-- Logs begin at Tue 2022-08-30 00:09:52 UTC, end at Tue 2022-08-30 01:32:26 UTC. -- Aug 30 00:47:44.183266 dv-cinder-815712-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 00:47:44.192217 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 00:47:44 2022] *** Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: nodename: dv-cinder-815712-11-iscsi Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: machine: x86_64 Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: clock source: unix Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: pcre jit disabled Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: detected number of CPU cores: 4 Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: current working directory: / Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: detected binary path: /usr/bin/uwsgi-core Aug 30 00:47:44.194244 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: your processes number limit is 47680 Aug 30 00:47:44.194832 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: your memory page size is 4096 bytes Aug 30 00:47:44.194832 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: detected max file descriptor number: 2048 Aug 30 00:47:44.194832 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: lock engine: pthread robust mutexes Aug 30 00:47:44.194832 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: thunder lock: enabled Aug 30 00:47:44.194832 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 00:47:44.194832 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 00:47:44.234085 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: Python main interpreter initialized at 0x55a2ddf4a9c0 Aug 30 00:47:44.234085 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: python threads support enabled Aug 30 00:47:44.234085 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: your server socket listen backlog is limited to 100 connections Aug 30 00:47:44.234085 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: your mercy for graceful operations on workers is 90 seconds Aug 30 00:47:44.234483 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 00:47:44.234559 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: *** Operational MODE: preforking *** Aug 30 00:47:44.234791 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: *** uWSGI is running in multiple interpreter mode *** Aug 30 00:47:44.234791 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: spawned uWSGI master process (pid: 81609) Aug 30 00:47:44.235003 dv-cinder-815712-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 00:47:44.241772 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: spawned uWSGI worker 1 (pid: 81620, cores: 1) Aug 30 00:47:44.241772 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: spawned uWSGI worker 2 (pid: 81621, cores: 1) Aug 30 00:47:44.241772 dv-cinder-815712-11-iscsi devstack@keystone.service[81609]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 00:47:48.640990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:48.640990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Aug 30 00:47:48.642394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:48.642394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Aug 30 00:47:48.655836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:48.655836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: from cryptography.utils import int_from_bytes Aug 30 00:47:48.656404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 00:47:48.656404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: from cryptography.utils import int_from_bytes Aug 30 00:47:48.692728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 00:47:48.692917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 00:47:48.693097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] command line args: [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 00:47:48.693370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 00:47:48.693549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ================================================================================ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 00:47:48.693834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.694088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_dir = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.694280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.694464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_source = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.694646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] debug = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.694846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.695214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.695395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.695653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.695906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.696141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.696323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_config_append = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.696504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.696734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_dir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.696854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.697108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.697343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.697591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.697863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.698047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.698225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.698471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.698719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.698957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.699194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.699428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.699681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.699990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.700163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.700438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.700935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.701123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] publish_errors = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.701357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.701589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.701791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] standard_threads = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_journal = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_json = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.704743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_stderr = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.705498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_syslog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.705498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.705498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.705498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.705498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.705868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.705868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.706014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.706254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.706442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 00:47:48.706523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.706640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 00:47:48.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.706836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] command line args: [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 00:47:48.706983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.707117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 00:47:48.707216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.707289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ================================================================================ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 00:47:48.707452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.707533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] admin_token = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.707686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.707770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config_dir = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.707926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.708011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.708091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] config_source = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.708167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.708271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] debug = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.708414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.708499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.708656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.708736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.708872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.708966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.709137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.709216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.709386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.709471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.709623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.709707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.709895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_config_append = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.709969 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.710045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.710157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.710243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_dir = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.710319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.710459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.710557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.710643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.710805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.710884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.711055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.711141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.711309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.711392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.711488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.711571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.711654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.711792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.711898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.712043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.712145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.712277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.712368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.712511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.712589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.712758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.712837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.713008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.713097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.713248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.713326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.713499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.713583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.713756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.713853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.713997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.714237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.714319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.714446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] publish_errors = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.714535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.714681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.714766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.714910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.714987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.715070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.715141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.715319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.715397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.715555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.715639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.715788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] standard_threads = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.715868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.716012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.716091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.716256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.716341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.716424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.716501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.716598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_journal = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.716715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.716794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_json = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.716936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_stderr = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.717021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.717098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] use_syslog = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.717170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.717332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 00:47:48.717410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.717867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.717952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.718958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.719087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.719199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.719312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.719458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.720171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.720368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.720494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.720631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.720984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.721160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.721265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.721430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.721614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.721758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.721908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.722978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.723904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.724872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.725851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.726930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.727919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.728937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.729896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.730997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.external = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.731957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.password = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.token = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.732966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.733938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.734895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.735013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.735240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.735747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.735834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.735907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.735979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.736883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.737907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.738946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.739928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.740990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.741979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.742979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.743985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.744970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.external = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.745981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.746062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.750579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.password = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] auth.token = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.751940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.752931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753049 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.753998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754822 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.754973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.755932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.756957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.757194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.757285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.757439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.757641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.757721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.757960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.758990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.759135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.759385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.759464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.759642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.759800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.759880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.760148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.760233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.760389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.760622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.760702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.760955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.761965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.762116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.762326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.762407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.762591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.762762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.762844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.763076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.763203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.763314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.763563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.763649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.763812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.764958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.765201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.765279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.765441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.765678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.765771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.765954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.766155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.766244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.767085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.767635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.767635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.767845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.767920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.767993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.768997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.769884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.770984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.771953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.772930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.773972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774822 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.774903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.775988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.776903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.777958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.778908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.779974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.780977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.781969 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.782174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.782284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.782549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.784931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.785974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.786985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.787987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.788957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.789934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.790026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.790172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.790252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.790345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 00:47:48.790425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.790661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.790912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.791160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.791411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.791652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.791893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.792132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.792381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.792656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.792891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.793137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.793406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.793670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.793943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.794198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.794443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.794687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.794924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.795163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.795402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.795640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.795888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.796126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.796374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.796612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.796861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.797099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.797346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.797603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.798044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.798476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 00:47:48.798654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81621) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 00:47:48.820852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.821080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.821295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.824233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.824472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.824708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.830359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.833915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.858941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.859203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.860974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.861212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.914496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.915940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.968800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.969082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.974008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:48.974249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.129860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.130142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.130311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.130530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.130671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.130808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.130956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.131094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.131226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.131226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.131485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.131616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.154575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.154807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.154960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.155890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.156169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.156301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.156432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.156575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.168201 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.169070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.213855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.214113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.214271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.214489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.214708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.214925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.215140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.223125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.223358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.223586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.223813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.224157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.224431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.225678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81621) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 00:47:49.232506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.232716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:49.233059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.233259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:49.233533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.233822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.234084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.234341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.235845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.236146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:49.236580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.236869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 00:47:49.237200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.237465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.237752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.238019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.247608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.247895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.248173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.248462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.249908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.250316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.250746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.251678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.256005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.259041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.259332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.259627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.259920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.260212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.260478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.260753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.261035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.261316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.261597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.261901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.263803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.268213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.268506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.268793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.269078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.269362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.269625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.269924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.270202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.270474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.270749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.271015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.285828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.287327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.287621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.292122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.292409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.292695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.294781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.296248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.296532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.299794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.304184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.304288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.304288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.304288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.304553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.304553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.304553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.304837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.305120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.305403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.305680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.307838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.308414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.308700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.312594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.312887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.313244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.314037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.314288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.316386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.316671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.316955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.323806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.324091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.327950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.331167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.331470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.332275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.332561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.335783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.336396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.338906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.339464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.339761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.341993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.342298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.342843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.343132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.343412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.343707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.344354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.347476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.350658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.350953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.351240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.351544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.351835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.352215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.359698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.360518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.360797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.361069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.367789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.368244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.368537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.368806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.369088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.369311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.370991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.371275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.373934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.374226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.374526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.374828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.375124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.376329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.378102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.379633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.384312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.384312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.384312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.384312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.384312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.384312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.386943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.387239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.387524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.387809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.388095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.388377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.388666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.394973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.395262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.395547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.395831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.396111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.396396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.396681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.404031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.407002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.410225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.411671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.411840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.413144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.413438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.413882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.414893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.417745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.419169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.420265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.420626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.420909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.421193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.421757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.425261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.425558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.425875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.426159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.428091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.430208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.432376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.432666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.432963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.433245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.433992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.434286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.434569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.441640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.441938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81621) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.442045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.442230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.442336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.442627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.442915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.443198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.443483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.443771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.444063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.444356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.444636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.444914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.445182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.445457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.445761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81621) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.448981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 00:47:49.449269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.449552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.449853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.450128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.450397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.450671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.450967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.451246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.451521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.451792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.452053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.452318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.452594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 00:47:49.479529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a2ddf4a9c0 pid: 81621 (default app) Aug 30 00:47:49.483119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd12f-de2e-4e26-a109-8c12d4143650 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:49.483411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd12f-de2e-4e26-a109-8c12d4143650 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:49.483696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fd12f-de2e-4e26-a109-8c12d4143650 None None] PATH_INFO: `/v3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:49.487035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55a2ddf4a9c0 pid: 81620 (default app) Aug 30 00:47:49.490510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Tue Aug 30 00:47:49 2022] GET /identity/v3/ => generated 255 bytes in 10 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:47:57.581460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.582015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.582132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.586448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:47:57 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:47:57.591806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:57.592516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:57.592621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:57.662994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: INFO dbcounter [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] Registered counter for database keystone Aug 30 00:47:57.684519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG oslo_db.sqlalchemy.engines [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81621) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 00:47:57.691613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writer thread running {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 00:47:57.876646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG passlib.handlers.bcrypt [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81621) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 00:47:57.877042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG passlib.handlers.bcrypt [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81621) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 00:47:57.935364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:47:57.936969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:47:58.135290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:47:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 544 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 00:47:58.621417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.693463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: INFO dbcounter [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] Registered counter for database keystone Aug 30 00:47:58.714254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG oslo_db.sqlalchemy.engines [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81620) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 00:47:58.721018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writer thread running {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 00:47:58.841438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f2c05-e2d3-47b5-bdf0-dec04c47d9b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.846606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.847252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.847549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.847869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.856529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:58.910471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 00:47:58.910816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a6b09-904c-4395-b570-898558fbafba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:58.920661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:47:58.927285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:47:58.942224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b51b5b9-1b90-4e0c-b85d-15ee9fb39ca3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:47:58.946728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:47:58.947858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:47:58.948142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:47:58.948440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] PATH_INFO: `/v3/projects/dae85e39d46d422d97a780b67b09cea1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:47:58.950092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] RBAC: Authorizing `identity:get_project(project_id=dae85e39d46d422d97a780b67b09cea1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:47:59.003717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 00:47:59.004069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1805a3df-bc46-4f2d-a1cc-0f58b7f5a7db admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:47:59.008760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Tue Aug 30 00:47:58 2022] GET /identity/v3/projects/dae85e39d46d422d97a780b67b09cea1 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:00.916232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adae32-8a6c-462e-91c8-e078fea51858 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.916749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adae32-8a6c-462e-91c8-e078fea51858 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.916858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0adae32-8a6c-462e-91c8-e078fea51858 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.920895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:00.926475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:00.926858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:00.927038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:00.947508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:00.949030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:00.957557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 00:48:01.231024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adae32-8a6c-462e-91c8-e078fea51858 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:01.243259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0adae32-8a6c-462e-91c8-e078fea51858 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:01.247720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5fc4a-976f-404d-8546-198d6aa4f76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:01.248319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5fc4a-976f-404d-8546-198d6aa4f76b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.248606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5fc4a-976f-404d-8546-198d6aa4f76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.248888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5fc4a-976f-404d-8546-198d6aa4f76b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.256773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5fc4a-976f-404d-8546-198d6aa4f76b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:01.258632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee5fc4a-976f-404d-8546-198d6aa4f76b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:01.268131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Tue Aug 30 00:48:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 00:48:01.273889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:01.283588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b5fa9-f319-48af-ad0b-05c501c5ae56 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:01.288322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c9f66-3c13-4a21-8c26-d540b23f51aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:01.288965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c9f66-3c13-4a21-8c26-d540b23f51aa admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:01.289249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c9f66-3c13-4a21-8c26-d540b23f51aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:01.289530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c9f66-3c13-4a21-8c26-d540b23f51aa admin admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:01.290889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c9f66-3c13-4a21-8c26-d540b23f51aa admin admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:01.292732 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c9f66-3c13-4a21-8c26-d540b23f51aa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:01.294289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Tue Aug 30 00:48:01 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:03.320682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298349ee-1046-4455-a7cd-19c5cf4626db None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.321286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800ce9e-75e2-44d6-8f9e-41c85243785d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.321750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298349ee-1046-4455-a7cd-19c5cf4626db None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.321840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800ce9e-75e2-44d6-8f9e-41c85243785d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.321918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298349ee-1046-4455-a7cd-19c5cf4626db None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.322007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800ce9e-75e2-44d6-8f9e-41c85243785d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.322617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:03.322739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:03.327776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.327897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.328063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.328176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.328909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.329022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.347214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:03.348499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:03.382663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81620) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 00:48:03.383048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81620) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 00:48:03.432447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:03.434086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:03.438998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 7/12] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:03.536486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 6/13] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:03.707938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.731959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.814575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c955849-d978-4bc4-8fde-5ba83b6a2ef5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.825245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:03.849637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.850296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.850577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.850857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.923121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:03.924989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:03.934324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10fa3-1bcb-4b4e-87ea-0627ba6149a3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:03.936665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.957651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:03.976100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:03.977187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:03.977480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:03.977788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:03.992622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:04.001506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:04.002710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:04.003772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:04.008653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.041450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 8/15] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:04.047875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.112459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b39e4-2ab3-4b54-87bc-3f56ef749967 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.146547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.147159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.147299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f7b6d-d2a8-464c-9953-0773b3d11e06 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.147439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.147714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.148383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:04.148686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.150143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.162538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 8/16] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:04.168210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.170889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.183078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.183787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.184031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.184280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.194928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.196416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.197254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:04.215855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:04.216773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:04.217878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:04.217878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:04.217878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:04.217878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:04.224597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.275420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf29bdb-c15b-402d-86d3-561a8cb347ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.315189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.315845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.316129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.316398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.317937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.319411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.321109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 9/18] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:04.326789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.333248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1511b5a9-2c6a-4e85-8807-6323266872ff None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.371642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.372264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.372550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.372837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.373486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:04.373817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.375259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.393394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 10/19] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:04.431667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.439740 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e93d24-3e73-493d-993f-95a71b720f33 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.479816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7e3c1b-0f45-4526-9105-eb20e7c81396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.480481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e3c1b-0f45-4526-9105-eb20e7c81396 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.480766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e3c1b-0f45-4526-9105-eb20e7c81396 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.481047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7e3c1b-0f45-4526-9105-eb20e7c81396 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.482259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e3c1b-0f45-4526-9105-eb20e7c81396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.483754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7e3c1b-0f45-4526-9105-eb20e7c81396 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.485515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 10/20] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:04.491598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:04.530259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:04.600317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd129c78-2f78-4e07-9a5b-238e0d3577e0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:04.637939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:04.638564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:04.638840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:04.639112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:04.639834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.domain.id=default, user.id=efbdbf875e6a427e9e8f14457c4de416) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:04.640146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:04.642479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56982212-27c2-45b8-8b32-e955b4a26acb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:04.653418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 11/21] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Aug 30 00:48:04 2022] GET /identity/v3/role_assignments?user.id=efbdbf875e6a427e9e8f14457c4de416&scope.domain.id=default&role.id=9d7628878120443f8b9950c60d7a8671 => generated 233 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:06.680825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0028be0-8864-40a8-a47b-f441cb01250e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.681805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0028be0-8864-40a8-a47b-f441cb01250e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.682287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0028be0-8864-40a8-a47b-f441cb01250e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.683824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 11/22] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:06.699542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6747d7-b837-4ea1-b488-0d5fa024e329 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.700061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6747d7-b837-4ea1-b488-0d5fa024e329 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.700136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6747d7-b837-4ea1-b488-0d5fa024e329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.726780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ca6747d7-b837-4ea1-b488-0d5fa024e329 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:06.728009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ca6747d7-b837-4ea1-b488-0d5fa024e329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:06.825906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 12/23] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:06.836173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9df94-b247-4a47-ad84-57655bb82f12 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.836747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9df94-b247-4a47-ad84-57655bb82f12 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.837225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9df94-b247-4a47-ad84-57655bb82f12 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.839036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:06.844836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baca4a-2d29-4209-bea3-fa2845fca774 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:06.845132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baca4a-2d29-4209-bea3-fa2845fca774 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:06.845417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baca4a-2d29-4209-bea3-fa2845fca774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:06.871749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c7baca4a-2d29-4209-bea3-fa2845fca774 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:06.873022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c7baca4a-2d29-4209-bea3-fa2845fca774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:06.909275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:06.972183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 13/25] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:07.273807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9df94-b247-4a47-ad84-57655bb82f12 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.314560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bc01-7ac8-4e34-b2f7-218a304efb7d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.314841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bc01-7ac8-4e34-b2f7-218a304efb7d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.315125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7bc01-7ac8-4e34-b2f7-218a304efb7d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.316458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 14/26] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:07.321838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.322129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.322411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.358961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:07.360751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:07.486903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9df94-b247-4a47-ad84-57655bb82f12 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.490434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 15/27] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:07.491529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.534808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.535529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.535966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.536386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.570998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.572740 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b975f3d7-fd6c-4bb7-b9cd-6eb5a346d4d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:48:07.601295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:07.602305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:48:07.603117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:48:07.603117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:48:07.603117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:07.605060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 13/28] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:07.606680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b538c-e89e-4826-b3e3-e78235701aad None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.607246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b538c-e89e-4826-b3e3-e78235701aad None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.607723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b538c-e89e-4826-b3e3-e78235701aad None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.609537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 14/29] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:07.617710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b538c-e89e-4826-b3e3-e78235701aad None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.622114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.637039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.641082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a2dc8-4cbe-4541-8ce9-fec49ba9613e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.708359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac64ede-2ff1-494e-b56c-01171a56d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.709838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64ede-2ff1-494e-b56c-01171a56d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.710342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64ede-2ff1-494e-b56c-01171a56d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.710826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac64ede-2ff1-494e-b56c-01171a56d2e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.726402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac64ede-2ff1-494e-b56c-01171a56d2e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2ac64ede-2ff1-494e-b56c-01171a56d2e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:07.734762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:07.735648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:07.736413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:07.739498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 16/30] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:07.741584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.742130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.742607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.764061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:07.766709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:07.817997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b538c-e89e-4826-b3e3-e78235701aad None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.844584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 17/31] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:07.845890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.857754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.858538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.858929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.859392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.860216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:07.860665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:07.862359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:07.886427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 15/32] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:07.887472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:07.935153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.958691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfd2b5-06d1-46b1-bbde-9cb96f7e874f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:07.971172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:07.997182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:07.997802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:07.998090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:07.998365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:07.999030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:07.999332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.000870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0d41b-b94b-44b6-b0d8-c505b488f4a0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:08.000983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.013856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 18/33] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:08.014682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:08.041520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:08.042532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.043015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.043510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.044486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.046100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.144268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc54cc9-5112-4295-ba60-a0ac2973f2a5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:08.181957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f1775-b9b0-4589-afba-8b57fdbc54f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:08.182638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f1775-b9b0-4589-afba-8b57fdbc54f6 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.182919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f1775-b9b0-4589-afba-8b57fdbc54f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.183196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f1775-b9b0-4589-afba-8b57fdbc54f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.183938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f1775-b9b0-4589-afba-8b57fdbc54f6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.186098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f1775-b9b0-4589-afba-8b57fdbc54f6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.198779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:08.329145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 16/34] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 30 00:48:07 2022] POST /identity/v3/roles => generated 219 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:08.330209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:08.447777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9d70-4a68-42a4-b691-e70d4ed405f4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:08.501129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:08.501849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.502139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.502442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.504911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.506594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.508475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 17/35] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:08 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:08.509518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:08.513188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:08.532643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 19/36] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:48:08 2022] POST /identity/v3/projects => generated 301 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:08.535136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce090e-b974-4c35-9a9b-8cf5fe8d54c7 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.535473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce090e-b974-4c35-9a9b-8cf5fe8d54c7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.535714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce090e-b974-4c35-9a9b-8cf5fe8d54c7 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.537035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 20/37] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:08.538241 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b7037-af8c-47a8-ac44-5c93fdb4179c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.538663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b7037-af8c-47a8-ac44-5c93fdb4179c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.538928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b7037-af8c-47a8-ac44-5c93fdb4179c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.540238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 21/38] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:08.541279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b7037-af8c-47a8-ac44-5c93fdb4179c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:08.642110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6752b82-90a8-4f31-8b66-38f03f1e85de None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:08.677589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b7037-af8c-47a8-ac44-5c93fdb4179c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:08.686993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbe23b-2609-490e-8dfb-bf5df4ccd270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:08.688260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe23b-2609-490e-8dfb-bf5df4ccd270 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.688546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe23b-2609-490e-8dfb-bf5df4ccd270 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.688826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe23b-2609-490e-8dfb-bf5df4ccd270 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.689515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbe23b-2609-490e-8dfb-bf5df4ccd270 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.690956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbe23b-2609-490e-8dfb-bf5df4ccd270 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.721210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2b7fea-6979-4769-970f-f6032f0b1c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:08.721949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b7fea-6979-4769-970f-f6032f0b1c28 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.722154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b7fea-6979-4769-970f-f6032f0b1c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.722434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2b7fea-6979-4769-970f-f6032f0b1c28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.723672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b7fea-6979-4769-970f-f6032f0b1c28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:08.725255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2b7fea-6979-4769-970f-f6032f0b1c28 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:08.726874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 22/39] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:08.728006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.728389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.728678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.749897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:08.752271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:08.850245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 18/40] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 30 00:48:08 2022] POST /identity/v3/roles => generated 225 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:08.851684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:08.851970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:08.852254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:08.877505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:08.879223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:08.939235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:08.952472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 23/41] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:08.953494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:08.960865 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:08.976252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 19/42] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:09.109297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab74cf4-a47f-4ca3-a3f0-25d50c87ceda None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:09.150041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:09.150720 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.151003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.151284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] PATH_INFO: `/v3/domains/default/users/efbdbf875e6a427e9e8f14457c4de416/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.153802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=efbdbf875e6a427e9e8f14457c4de416, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:09.155472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:09.272288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:09.351051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 24/43] 10.222.0.52 () {58 vars in 1361 bytes} [Tue Aug 30 00:48:08 2022] PUT /identity/v3/domains/default/users/efbdbf875e6a427e9e8f14457c4de416/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:09.351907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:09.434559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bd7479-a8b4-4aea-9238-a4a25c407abb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:09.444020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:09.495013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ebc02a-58af-48d9-84df-a5d7d51b8e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:09.495659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebc02a-58af-48d9-84df-a5d7d51b8e7a None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.495943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebc02a-58af-48d9-84df-a5d7d51b8e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.496221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ebc02a-58af-48d9-84df-a5d7d51b8e7a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.496999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebc02a-58af-48d9-84df-a5d7d51b8e7a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:09.498457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ebc02a-58af-48d9-84df-a5d7d51b8e7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:09.534768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3885207-d6fa-4698-a40c-1a63f68018d7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:09.583244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:09.583906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.584200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.584476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.585641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:09.587234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:09.589091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 25/44] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:09.594256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 20/45] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 30 00:48:09 2022] POST /identity/v3/roles => generated 223 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:09.595747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:09.736618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bd03d6-c9a4-4ccf-9faa-74ff4709ab8a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:09.764204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:09.788358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d781b-2f27-491e-9c81-12fc8ebb5ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:09.789347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d781b-2f27-491e-9c81-12fc8ebb5ad8 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:09.789696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d781b-2f27-491e-9c81-12fc8ebb5ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:09.790016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d781b-2f27-491e-9c81-12fc8ebb5ad8 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:09.791078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d781b-2f27-491e-9c81-12fc8ebb5ad8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:09.793464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d781b-2f27-491e-9c81-12fc8ebb5ad8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:09.912537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 26/46] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:48:09 2022] POST /identity/v3/projects => generated 312 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:11.373194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90de3f-fe87-4a8b-bb06-250ea15fc704 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.373713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90de3f-fe87-4a8b-bb06-250ea15fc704 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.373906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f90de3f-fe87-4a8b-bb06-250ea15fc704 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.375796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 21/47] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:11.381035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1fabe-511a-4a15-9e5b-35631d122389 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.381423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1fabe-511a-4a15-9e5b-35631d122389 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.381597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb1fabe-511a-4a15-9e5b-35631d122389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.403501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1eb1fabe-511a-4a15-9e5b-35631d122389 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:11.404740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1eb1fabe-511a-4a15-9e5b-35631d122389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:11.487237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 27/48] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:11.755830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90de3f-fe87-4a8b-bb06-250ea15fc704 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.815032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:11.864761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f90de3f-fe87-4a8b-bb06-250ea15fc704 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:11.869304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a64f73-acf0-4e8e-892d-c5ca44cb6a59 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.869600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a64f73-acf0-4e8e-892d-c5ca44cb6a59 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.869919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a64f73-acf0-4e8e-892d-c5ca44cb6a59 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.871253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 28/49] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:11.876976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.877262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.877546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.895635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:11.896824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:11.899972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:11.900625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:11.900914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:11.901200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:11.913152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:11.918466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:11.919271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:11.920048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 22/50] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:11.925999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:11.965963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 29/51] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:12.028103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460217ca-e24c-4775-9c28-d791712a84b9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.041209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.062149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.062869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.063151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.064775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.064775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:12.064775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.065713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.074162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 23/52] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:12.079922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.111691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.179752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e95a9-7291-4107-ac26-7715bd68f48f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.213358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.214030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.214316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.214592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.215712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.217440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.219910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 30/53] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:12.225675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.233650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.328347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a9c1c8-0267-4258-b856-7f7229cb20c1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.331285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.334901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7571f060-d26e-4ae5-b2d3-f0a129482836 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.364062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.364707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.365055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.365336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.366606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.368061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.369634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 24/54] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:12.370083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.370762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.371008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.371286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.372419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.373888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.375695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.376128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 31/55] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:12.381174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:12.421205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:12.479403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad66f3-8d41-42e3-aa04-47a01d99b7d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.488157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a195e57-0756-4daa-9142-c2fcb9e409e3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:12.515997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.516718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.517010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.517294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.518060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.domain.id=default, user.id=efbdbf875e6a427e9e8f14457c4de416) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:12.518375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.519820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56464578-5532-4497-a9ba-5eeef0222ad1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.522540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714705ef-3516-4cc1-8d59-1f159d160747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:12.523170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714705ef-3516-4cc1-8d59-1f159d160747 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:12.523452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714705ef-3516-4cc1-8d59-1f159d160747 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:12.523727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714705ef-3516-4cc1-8d59-1f159d160747 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:12.524374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714705ef-3516-4cc1-8d59-1f159d160747 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:12.525840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714705ef-3516-4cc1-8d59-1f159d160747 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:12.528432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 25/56] 10.222.0.52 () {56 vars in 1322 bytes} [Tue Aug 30 00:48:12 2022] GET /identity/v3/role_assignments?user.id=efbdbf875e6a427e9e8f14457c4de416&scope.domain.id=default&role.id=9d7628878120443f8b9950c60d7a8671 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:12.643038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 32/57] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Aug 30 00:48:12 2022] POST /identity/v3/projects => generated 298 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:48:14.569305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fecf8-befd-462c-8c43-d8db2c9173cd None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.569828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fecf8-befd-462c-8c43-d8db2c9173cd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.569909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7fecf8-befd-462c-8c43-d8db2c9173cd None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.571238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 26/58] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:14.576530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:14.576948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:14.577102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:14.596079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:14.597286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:14.611045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:14.664411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 33/59] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:14.938364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fecf8-befd-462c-8c43-d8db2c9173cd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.147360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7fecf8-befd-462c-8c43-d8db2c9173cd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.154928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:15.181776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe342ad-93ca-4163-8979-91b72fb3b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.182428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe342ad-93ca-4163-8979-91b72fb3b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.182717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe342ad-93ca-4163-8979-91b72fb3b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.183031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe342ad-93ca-4163-8979-91b72fb3b7f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.184293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe342ad-93ca-4163-8979-91b72fb3b7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.186072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe342ad-93ca-4163-8979-91b72fb3b7f5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.188493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 27/60] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:15.193987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:15.290148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c370abe-6f02-417f-901f-aa48292a4d8d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:15.323438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24a7596-91a1-4ca8-a536-a713906b4f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:15.324042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7596-91a1-4ca8-a536-a713906b4f9b None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:15.324323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7596-91a1-4ca8-a536-a713906b4f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:15.324595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24a7596-91a1-4ca8-a536-a713906b4f9b None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:15.325235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7596-91a1-4ca8-a536-a713906b4f9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:15.326713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24a7596-91a1-4ca8-a536-a713906b4f9b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:15.459235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 34/61] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:15 2022] POST /identity/v3/users => generated 275 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:18.500851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf4257-6451-4826-98b2-6072610e1571 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.501817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf4257-6451-4826-98b2-6072610e1571 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.502300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf4257-6451-4826-98b2-6072610e1571 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.503919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 28/62] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.509337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c346-62d2-4ff1-ac6b-5ec3fbc5cb7f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.510207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c346-62d2-4ff1-ac6b-5ec3fbc5cb7f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.510658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6c346-62d2-4ff1-ac6b-5ec3fbc5cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.522644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:18.539468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a8b6c346-62d2-4ff1-ac6b-5ec3fbc5cb7f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:18.540805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a8b6c346-62d2-4ff1-ac6b-5ec3fbc5cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:18.647814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 35/63] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:18.722594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616d647-494d-448c-9ca5-b2a6541f648f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.722897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616d647-494d-448c-9ca5-b2a6541f648f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.723161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616d647-494d-448c-9ca5-b2a6541f648f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.724763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 29/64] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.730160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eec88-cc48-45ef-8b84-861114805b89 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.730547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eec88-cc48-45ef-8b84-861114805b89 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.730879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86eec88-cc48-45ef-8b84-861114805b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.755337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f86eec88-cc48-45ef-8b84-861114805b89 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:18.756512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f86eec88-cc48-45ef-8b84-861114805b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:18.828769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 36/65] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:18.896001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a32c9-0be4-4acc-a646-838ac6af5d0d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.896514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a32c9-0be4-4acc-a646-838ac6af5d0d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.897010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a32c9-0be4-4acc-a646-838ac6af5d0d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.899850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 30/66] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:18.909746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:18.910267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:18.910763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:18.957573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:18.959645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:19.100817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.107728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a32c9-0be4-4acc-a646-838ac6af5d0d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.127616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 37/67] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:19.170517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.321657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a32c9-0be4-4acc-a646-838ac6af5d0d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.344968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ca9887-2f9a-4e07-ba9a-e649f058f8aa None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.366369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbc7e94-d490-410a-967e-01dc40a4f1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.366989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc7e94-d490-410a-967e-01dc40a4f1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.367226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc7e94-d490-410a-967e-01dc40a4f1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.367507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbc7e94-d490-410a-967e-01dc40a4f1e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.379449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbc7e94-d490-410a-967e-01dc40a4f1e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9bbc7e94-d490-410a-967e-01dc40a4f1e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:19.384380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:19.385264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:19.386095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:19.389931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 31/68] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:19.395479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf381d4-6e73-4a76-b5fb-2264743ea479 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.396064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf381d4-6e73-4a76-b5fb-2264743ea479 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.396495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf381d4-6e73-4a76-b5fb-2264743ea479 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.399039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 32/69] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:19.400701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f97010-189b-4a17-a454-3c60dde1f59e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.401200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f97010-189b-4a17-a454-3c60dde1f59e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.401681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f97010-189b-4a17-a454-3c60dde1f59e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.404160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 33/70] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:19.405739 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f97010-189b-4a17-a454-3c60dde1f59e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.408122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acfe5c3-bfaa-4894-a154-2d03a3b26f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.409028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfe5c3-bfaa-4894-a154-2d03a3b26f84 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.409525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfe5c3-bfaa-4894-a154-2d03a3b26f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.410047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acfe5c3-bfaa-4894-a154-2d03a3b26f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.423122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acfe5c3-bfaa-4894-a154-2d03a3b26f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-6acfe5c3-bfaa-4894-a154-2d03a3b26f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:19.428088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:19.428951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:19.429755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 38/71] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:19.431455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f808cb-9c4a-45d5-9801-19f13924f3f9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.431455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f808cb-9c4a-45d5-9801-19f13924f3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.431650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f808cb-9c4a-45d5-9801-19f13924f3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.449362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.452527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-69f808cb-9c4a-45d5-9801-19f13924f3f9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:19.454058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-69f808cb-9c4a-45d5-9801-19f13924f3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:19.513721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.524605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f97010-189b-4a17-a454-3c60dde1f59e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.532183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 39/72] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:19.543514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.544450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.544962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.563835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.564345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.564637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.564911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.565587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:19.565994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.567392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.572768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:19.574163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:19.581782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 34/73] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.581782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.651596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 40/74] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:19.652442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.703717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.712851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781c0e36-d5e7-4c56-8a55-88053cf3144a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.767395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.768364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.768833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.769039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57332f64-3ea9-4250-a57a-6e8adb990973 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.769477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.770377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:19.771143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.773673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.786159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.786460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 35/75] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.788451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.816644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.818427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.819168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.819865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.832631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:19.837542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:19.838530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 41/76] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:19.842600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.909118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5bb209-8221-46a1-a050-d2bee3aac361 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.947309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.947960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.948226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.948510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.952966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c122b0b6-1c14-4647-a1f2-dd405f7984b5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:19.964723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.966235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.968150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 36/77] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.969369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:19.987667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:19.988315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:19.988960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:19.989615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:19.989945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:19.992500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:19.992626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:19.998942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 42/78] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:19.998942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.040679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.082233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77a5ef-22a9-4f0c-bc8e-c099914881fc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.106765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd03a-1cea-4290-b44b-453ac0c93636 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.120994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.121786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.122167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.122537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.134265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:20.139335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:20.140484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:20.141418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 37/79] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:20.142096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.154050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.154697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.154980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.155263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.155975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:20.156281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.157666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.166486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 43/80] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.167240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.225255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.249253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215665d-8d90-4b0c-bb52-e8d4f2cc7d55 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.269472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2fb71-27ff-44c2-a558-2d72977434b9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.272227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.285620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.286287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.286577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.286854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.287986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.289448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.291037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 38/81] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.291812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.305588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.306246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.306536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.306821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.378592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.380072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.398195 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0331aba-ff08-48a3-99d9-d48c536b0d3f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.438287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.438832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.439118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.439401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.455226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.497973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 44/82] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:20.498833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.502951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.513038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.514613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.600482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66307100-13e3-4bab-8eb7-f3d2a20b2bd2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.617415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 39/83] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:20.618344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.635788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.636402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.636685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.636964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.637626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:20.637963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.639357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.648115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 45/84] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.648115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.671581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.716348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.727002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9bedc-5cc6-4ac0-bb78-79a51a0d3cc1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.752337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f1a777-3d89-41b3-bf83-19b719c110b4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.763016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.764088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.764088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.764281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.766234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.767922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.769524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 40/85] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.770330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.796872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.797552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.797883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.798167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.798847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.800302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.873336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a7e9f-0af3-4184-bd02-84674a21f070 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:20.886360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:20.898131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 46/86] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:20 2022] POST /identity/v3/users => generated 289 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:20.902712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:20.908795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:20.910275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:20.910503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:20.910798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:20.912131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:20.913697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:20.915499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 41/87] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:20.916330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.033710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca7a09-cfb0-4d9e-b0b6-e510d22cefc9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.047568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2ddce-ca75-40d7-9220-93f96abeb313 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.066499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.082471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.085530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.085530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.085530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.085530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.086433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.087327 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.088267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.088380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 47/88] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:21.088452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.088523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.089202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.089711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.091180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.093765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 42/89] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:20 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:21.094854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.158968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.224457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8240f76-c2c2-4ca7-9775-8b0ece490729 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.224810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e508-0c6b-4829-a2a3-38a7202daeff None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.272173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.272954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.273259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.273357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.273546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.273957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.274298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.274394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=ffd05526cecb4b7fb2137f53df2eeae7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:21.274575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.274675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.276139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.284507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 43/90] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/role_assignments?user.id=ffd05526cecb4b7fb2137f53df2eeae7&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:21.285901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.345836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.359430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.361277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.427417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a008643-cc0c-4cd8-a86c-827d842257fc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.446239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.497550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.498215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.498504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.498789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.499612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=efbdbf875e6a427e9e8f14457c4de416) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:21.499947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.501479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.512580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 44/91] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/role_assignments?user.id=efbdbf875e6a427e9e8f14457c4de416&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:21.513714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.601298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 48/92] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:21.609070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.768105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426059a3-aeea-45e1-8ca4-b7b19f8dda51 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.787471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.807244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0938d95a-151c-4a6c-98d1-115de69939a1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:21.838015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.838639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.838923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.839197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.878640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:21.879204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:21.879484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:21.879757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:21.886560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.888058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:21.890224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 49/93] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:21.896122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:21.926288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:21.962969 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:21.964457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.094550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 45/94] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:22.100073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.146079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910d537-5a3b-4171-8a79-6734b308509b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.186961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.187587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.188021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.188300 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.189035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] RBAC: Adding query filter params (role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=ffd05526cecb4b7fb2137f53df2eeae7) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:22.189345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.191361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.197418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 50/95] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:21 2022] GET /identity/v3/role_assignments?user.id=ffd05526cecb4b7fb2137f53df2eeae7&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=d0854046eb544eab8092d0a773a97724 => generated 259 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:22.252865 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.294180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04199cf1-227f-4759-9212-554ef5a22ccd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.348208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675ad4a-3d5d-4e83-9edb-53305440860e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.348879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675ad4a-3d5d-4e83-9edb-53305440860e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.349163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675ad4a-3d5d-4e83-9edb-53305440860e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.349445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675ad4a-3d5d-4e83-9edb-53305440860e None admin] PATH_INFO: `/v3/projects/457097532b824655a88961b47d5ce031` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.350700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675ad4a-3d5d-4e83-9edb-53305440860e None admin] RBAC: Authorizing `identity:get_project(project_id=457097532b824655a88961b47d5ce031)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.352176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0675ad4a-3d5d-4e83-9edb-53305440860e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.353893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 46/96] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/projects/457097532b824655a88961b47d5ce031 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:22.360835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:22.401840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:22.509301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d66a06e-aac1-4e8e-a99a-50f2b3441f09 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:22.554337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:22.555011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:22.555298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:22.555573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:22.556311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=457097532b824655a88961b47d5ce031, user.id=ffd05526cecb4b7fb2137f53df2eeae7) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:22.556622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:22.558095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e50221-484a-42b1-9c9e-d41cc48872b1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:22.566852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 51/97] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:22 2022] GET /identity/v3/role_assignments?user.id=ffd05526cecb4b7fb2137f53df2eeae7&scope.project.id=457097532b824655a88961b47d5ce031&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:23.475237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dc41-b2f2-41b3-a4ac-c3907f6ae49c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.475782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dc41-b2f2-41b3-a4ac-c3907f6ae49c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.475782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497dc41-b2f2-41b3-a4ac-c3907f6ae49c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.477133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 47/98] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:23.488135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6402-f028-4c9c-b44c-15a717316d11 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.488896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6402-f028-4c9c-b44c-15a717316d11 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.489392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb6402-f028-4c9c-b44c-15a717316d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.514676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a3eb6402-f028-4c9c-b44c-15a717316d11 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:23.516404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a3eb6402-f028-4c9c-b44c-15a717316d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:23.608447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 52/99] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:23.632824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff75c8-c8d5-4f90-b35f-6dc59ded7873 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.633142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff75c8-c8d5-4f90-b35f-6dc59ded7873 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.633403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ff75c8-c8d5-4f90-b35f-6dc59ded7873 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.634789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 48/100] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:23.644578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:23.644877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:23.645168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:23.670408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:23.672307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:23.684676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:23.764197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 53/101] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:24.197607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff75c8-c8d5-4f90-b35f-6dc59ded7873 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:24.198777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:24.320250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fb9e4-cd61-41a6-88dd-5ee8ccb1fd84 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:24.349705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ff75c8-c8d5-4f90-b35f-6dc59ded7873 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:24.378940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:24.380051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.381918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.383727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.385117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:24.389150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:24.394210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 54/102] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:24.398651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:24.426420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:24.427487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.428204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.429526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.430147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:24.457074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:24.466866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:24.467766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:24.468857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 49/103] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:24.479292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:24.486860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:24.579734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9edee-efb8-4424-ae80-4e96cf6ea2ca None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:24.626459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44762fa5-c71e-4d52-b523-eb7d0c1b4a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:24.627372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44762fa5-c71e-4d52-b523-eb7d0c1b4a57 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.627865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44762fa5-c71e-4d52-b523-eb7d0c1b4a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.628345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44762fa5-c71e-4d52-b523-eb7d0c1b4a57 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.629207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44762fa5-c71e-4d52-b523-eb7d0c1b4a57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:24.630919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44762fa5-c71e-4d52-b523-eb7d0c1b4a57 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:24.638622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fdd68-0139-4fe0-a617-d15cebc0d9e8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:24.679412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:24.680032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.680314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.680589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.681320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:24.681626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:24.683135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6bed5-f45a-4ef7-907a-fbb1fb522e25 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:24.693198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 50/104] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:24.697918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3000b-4192-45ce-a571-51e31252ef1d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.697918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3000b-4192-45ce-a571-51e31252ef1d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.697918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3000b-4192-45ce-a571-51e31252ef1d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.697918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 51/105] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:24.697918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6741d-23ac-40cd-9bbd-f7cf4a5470ee None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.697918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6741d-23ac-40cd-9bbd-f7cf4a5470ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.706210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6741d-23ac-40cd-9bbd-f7cf4a5470ee None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.706210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 52/106] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:24.706210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6741d-23ac-40cd-9bbd-f7cf4a5470ee None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:24.751543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 55/107] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:48:24 2022] POST /identity/v3/projects => generated 302 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:24.752419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ed32-dfe7-4608-adda-e5936d9c13cd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.752911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ed32-dfe7-4608-adda-e5936d9c13cd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.754021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359ed32-dfe7-4608-adda-e5936d9c13cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.775621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:24.781553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9359ed32-dfe7-4608-adda-e5936d9c13cd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:24.783458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9359ed32-dfe7-4608-adda-e5936d9c13cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:24.871737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:24.873869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 56/108] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:24.878491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.878642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.878930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.883844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6741d-23ac-40cd-9bbd-f7cf4a5470ee None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:24.916665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:24.918287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:24.941959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:24.942677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:24.942970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:24.943257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] PATH_INFO: `/v3/users/8cca739a010e4fdfa7d26314242982dd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:24.962465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] RBAC: Authorizing `identity:get_user(user_id=8cca739a010e4fdfa7d26314242982dd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:24.964395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:24.966654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 53/109] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:24 2022] GET /identity/v3/users/8cca739a010e4fdfa7d26314242982dd => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:24.973227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.021181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 57/110] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:25.107118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52b072-f514-4d27-82af-631471c9f47a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.144837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:25.156276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.156955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.161675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.161675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.209337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.243697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:25.245179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:25.357594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a8067-6af0-4f80-bfc6-c4bd7deb38e1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.386190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:25.396044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59de404-4654-478b-8946-8fd5c7362e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.396697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59de404-4654-478b-8946-8fd5c7362e83 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.396984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59de404-4654-478b-8946-8fd5c7362e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.397264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59de404-4654-478b-8946-8fd5c7362e83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.409699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59de404-4654-478b-8946-8fd5c7362e83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c59de404-4654-478b-8946-8fd5c7362e83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:25.415684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:25.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:25.417868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 58/111] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:25.418582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f97e0-9823-49e0-9eb3-dd1a6cef2da9 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.418582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f97e0-9823-49e0-9eb3-dd1a6cef2da9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.418701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f97e0-9823-49e0-9eb3-dd1a6cef2da9 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.420035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 59/112] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:25.426076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f97e0-9823-49e0-9eb3-dd1a6cef2da9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.448154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 54/113] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:48:25.449274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.490655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:25.556304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f97e0-9823-49e0-9eb3-dd1a6cef2da9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.574022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c025d7-bdee-4674-a2b1-756ea693ccdd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.599078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570082d7-6a43-4981-9e7e-6474e8b86bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.599730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570082d7-6a43-4981-9e7e-6474e8b86bc9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.600021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570082d7-6a43-4981-9e7e-6474e8b86bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.600302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570082d7-6a43-4981-9e7e-6474e8b86bc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.613664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570082d7-6a43-4981-9e7e-6474e8b86bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:25.613978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.614605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.614894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.615185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.615859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:25.616171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:25.617555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-570082d7-6a43-4981-9e7e-6474e8b86bc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:25.618542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:25.619332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:25.620122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 60/114] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:25.626788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37ed21-41f3-4097-a980-36e455813207 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.627082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37ed21-41f3-4097-a980-36e455813207 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.627355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b37ed21-41f3-4097-a980-36e455813207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.629946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 55/115] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:25.634917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.648637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2b37ed21-41f3-4097-a980-36e455813207 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:25.649903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2b37ed21-41f3-4097-a980-36e455813207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:25.697660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:25.723333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 61/116] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:25.740458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b41f2-e0dc-4eb2-856c-0e6bd2b786ba None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.740458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b41f2-e0dc-4eb2-856c-0e6bd2b786ba None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.740458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282b41f2-e0dc-4eb2-856c-0e6bd2b786ba None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.740458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 62/117] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:25.740458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b41f2-e0dc-4eb2-856c-0e6bd2b786ba None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.758614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:25.776555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1fc67d-c313-406a-8f6f-b7684136bf82 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.822351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.823001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.823290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.823569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.824730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:25.826293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:25.828480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 56/118] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:25 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:25.829283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:25.871768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282b41f2-e0dc-4eb2-856c-0e6bd2b786ba None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.919161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.920198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.920489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.921084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.921798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:25.922112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:25.924292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a4d11e-48c3-46fb-8ce3-de328eaeafb5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:25.936544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 63/119] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:25.938017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.938304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.938627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.945649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea907b-465c-4bee-88d5-68c6ad6ad30b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:25.958232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:25.959484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:25.991502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:25.993635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:25.994695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:25.995192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:25.995617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:25.997107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:25.999297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.001110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 57/120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:25 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.005494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.038780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 64/121] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:26.039137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.040144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.146983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24ca1ad-98ea-43f1-bcdd-abff99ac47c2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.148628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473eea5-0549-4e10-8037-e43a7efbbaa8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.191240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.192340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.192924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.193350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.194196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.194888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.195091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.195308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.195653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.196456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] RBAC: Adding query filter params (role.id=0ed49d44afa7491f8cf2b6f367dfcded, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=8cca739a010e4fdfa7d26314242982dd) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:26.196837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.197333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.198457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.199331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 65/122] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.200239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.209375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 58/123] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/role_assignments?user.id=8cca739a010e4fdfa7d26314242982dd&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=0ed49d44afa7491f8cf2b6f367dfcded => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.210433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.271839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.311369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.329916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca92302-af85-4a9e-a933-8c6cd4a58850 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.370103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.370801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.371298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.371426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.384220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:26.387861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afb6a35-d93b-481f-a0dd-f08b95e0db7d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:26.389373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:26.390592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:26.391393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 59/124] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:26.392020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.435828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.436157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.436429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.437337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.439029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.441097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.442445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 66/125] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.443441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.524267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104420b7-5900-48c5-b64a-41bc14e316a0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.572474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.574133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.575003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.575520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.577000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b2eeba-2f22-4e31-9dcc-33cdbed2a0e9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.578182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.582199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.582554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.587121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 60/126] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.588191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.609495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.632593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.634066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.635293 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.635726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.650112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:26.656969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:26.658060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:26.659007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:26.662958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 67/127] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:26.664451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.710237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b07d84-1994-4e04-81a5-130fcf1a9e8a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.749841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.750674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.751079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.751589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.752333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:26.752809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.754440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.763784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 61/128] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:26.769405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:26.811019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bb36f-c8f8-421b-ba20-991e6b1f4fa0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.855950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:26.861304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.862628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.863330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.866548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/efbdbf875e6a427e9e8f14457c4de416/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.866548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] RBAC: Authorizing `identity:create_grant(project_id=27f6325081de4a4bbd73323b17cd130a, user_id=efbdbf875e6a427e9e8f14457c4de416, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.867855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.881538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482091b-698b-44a1-8d4c-b8f1bc9d1052 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:26.924217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:26.924894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:26.925177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:26.925466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/ffd05526cecb4b7fb2137f53df2eeae7/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:26.927409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f6325081de4a4bbd73323b17cd130a, user_id=ffd05526cecb4b7fb2137f53df2eeae7, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:26.928895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:26.973845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 68/129] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:26 2022] PUT /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/efbdbf875e6a427e9e8f14457c4de416/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:26.973845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.006576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.093694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 62/130] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:26 2022] PUT /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/ffd05526cecb4b7fb2137f53df2eeae7/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:27.094870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.111819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60658b35-5972-4df8-bec6-66f71288963f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.179913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.180625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.180919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.181283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.182012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:27.182326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.183793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58543e88-621f-4b15-b804-e820bb638fbe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.201270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 69/131] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.203587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e723-7998-456a-b0a1-3ec92d0c4def None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.204015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e723-7998-456a-b0a1-3ec92d0c4def None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.204447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56e723-7998-456a-b0a1-3ec92d0c4def None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.206700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 70/132] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:27.212091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56e723-7998-456a-b0a1-3ec92d0c4def None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.276962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09a68e-5d3a-4bdd-8e75-8ca216f975e9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.317893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2aa27f-4925-44ac-8b1b-9b34d6eceab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.318549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aa27f-4925-44ac-8b1b-9b34d6eceab1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.318834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aa27f-4925-44ac-8b1b-9b34d6eceab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.319114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2aa27f-4925-44ac-8b1b-9b34d6eceab1 None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.322409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2aa27f-4925-44ac-8b1b-9b34d6eceab1 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.322409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2aa27f-4925-44ac-8b1b-9b34d6eceab1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.322596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.323544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 63/133] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.324748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.325035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.325424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.333477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.343727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56e723-7998-456a-b0a1-3ec92d0c4def None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.350309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:27.352068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:27.382817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.383144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.383393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.383682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.384863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.386428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.388494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 71/134] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.389428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.448643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 64/135] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:27.450061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.538382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf1721-aa1e-4ea8-a548-20ccb3e3922e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.585673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.586343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.586684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.586909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.588099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.595067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.596818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 72/136] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.622783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.669720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626d075-bb95-463b-a30a-fae8cbeb2025 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.692825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.715354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.716006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.716554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.716842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] PATH_INFO: `/v3/projects/457097532b824655a88961b47d5ce031` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.719164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] RBAC: Authorizing `identity:get_project(project_id=457097532b824655a88961b47d5ce031)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.720649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.722364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 65/137] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/projects/457097532b824655a88961b47d5ce031 => generated 312 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:27.731538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:27.736865 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:27.838568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf727da-73f7-4340-b62f-fb788630be00 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.855873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8904b94-fd49-4c39-a414-8c93a6d0e39c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:27.897207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.897926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.898218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.898504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] PATH_INFO: `/v3/projects/457097532b824655a88961b47d5ce031/users/ffd05526cecb4b7fb2137f53df2eeae7/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.899916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed32787-336c-4759-b15f-33671302190c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:27.900587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed32787-336c-4759-b15f-33671302190c None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:27.900867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed32787-336c-4759-b15f-33671302190c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:27.901143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed32787-336c-4759-b15f-33671302190c None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/ffd05526cecb4b7fb2137f53df2eeae7/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:27.903935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] RBAC: Authorizing `identity:create_grant(project_id=457097532b824655a88961b47d5ce031, user_id=ffd05526cecb4b7fb2137f53df2eeae7, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.904924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed32787-336c-4759-b15f-33671302190c None admin] RBAC: Authorizing `identity:create_grant(project_id=27f6325081de4a4bbd73323b17cd130a, user_id=ffd05526cecb4b7fb2137f53df2eeae7, role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:27.907646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.910073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed32787-336c-4759-b15f-33671302190c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:27.991046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 66/138] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:27 2022] PUT /identity/v3/projects/457097532b824655a88961b47d5ce031/users/ffd05526cecb4b7fb2137f53df2eeae7/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:27.992055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:28.019940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 73/139] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:27 2022] PUT /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/ffd05526cecb4b7fb2137f53df2eeae7/roles/d0854046eb544eab8092d0a773a97724 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:28.091925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.154639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839a7e0-790a-4797-9855-8b176f4d30af None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.215396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e1d13-d7b8-4772-b9be-3f277054381d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.216785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e1d13-d7b8-4772-b9be-3f277054381d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.217289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e1d13-d7b8-4772-b9be-3f277054381d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.217838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e1d13-d7b8-4772-b9be-3f277054381d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.219480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e1d13-d7b8-4772-b9be-3f277054381d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:28.221439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e1d13-d7b8-4772-b9be-3f277054381d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:28.223500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 67/140] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:28.229297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed32787-336c-4759-b15f-33671302190c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:28.343091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:28.367537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed32787-336c-4759-b15f-33671302190c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:28.412315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9831db-27ff-4417-ae83-480091397bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:28.412947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9831db-27ff-4417-ae83-480091397bd6 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.413231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9831db-27ff-4417-ae83-480091397bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.413512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9831db-27ff-4417-ae83-480091397bd6 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.414297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9831db-27ff-4417-ae83-480091397bd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:28.415790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9831db-27ff-4417-ae83-480091397bd6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:28.544454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 74/141] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:28 2022] POST /identity/v3/users => generated 283 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:28.669319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef86242-088d-4793-ac1b-64cdb8b5b06f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.670043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef86242-088d-4793-ac1b-64cdb8b5b06f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.671014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef86242-088d-4793-ac1b-64cdb8b5b06f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.672792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 68/142] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:28.678581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:28.679183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:28.679663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:28.700175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:28.702337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:28.799210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 75/143] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:29.323244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef86242-088d-4793-ac1b-64cdb8b5b06f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:29.470268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef86242-088d-4793-ac1b-64cdb8b5b06f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:29.505137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:29.516274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:29.516929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.517250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.517539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.529577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:29.535147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:29.536030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:29.537216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 69/144] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:29.542375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:29.650150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:29.688683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908734e4-0fa4-40a6-b306-746c88391820 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:29.731263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:29.731992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.732143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.732416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.733103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:29.733443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:29.735057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48f64a-14f3-475b-b237-3ed24497c1ff None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:29.744069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 76/145] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:29.750128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:29.797464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d5146-5cc5-4bec-8968-f4dda042ae0c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.797837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d5146-5cc5-4bec-8968-f4dda042ae0c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.798047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259d5146-5cc5-4bec-8968-f4dda042ae0c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.799604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 77/146] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:29.805149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.805441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.805723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.842062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:29.843875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:29.876823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34479c4f-5e70-482e-854b-476c13694d3c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:29.913480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:29.914527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:29.914950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:29.915369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] PATH_INFO: `/v3/users/8cca739a010e4fdfa7d26314242982dd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:29.916659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] RBAC: Authorizing `identity:get_user(user_id=8cca739a010e4fdfa7d26314242982dd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:29.918291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:29.920038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 70/147] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:29 2022] GET /identity/v3/users/8cca739a010e4fdfa7d26314242982dd => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:29.926303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:29.994126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:29.995086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 78/148] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:30.034880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e2b33-72da-457e-9774-5b05168fe9d2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.072941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.073944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.074375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.074785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.076062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:30.077790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:30.080101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 71/149] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:29 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:30.086093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.182980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:30.234053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006a2033-0050-4347-a5b8-ce33e1e65f1d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.270519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8d48b-2ebc-4d1d-a1ed-070362c6e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.271201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8d48b-2ebc-4d1d-a1ed-070362c6e0d4 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.271484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8d48b-2ebc-4d1d-a1ed-070362c6e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.271762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8d48b-2ebc-4d1d-a1ed-070362c6e0d4 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/8cca739a010e4fdfa7d26314242982dd/roles/0ed49d44afa7491f8cf2b6f367dfcded` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.273751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8d48b-2ebc-4d1d-a1ed-070362c6e0d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f6325081de4a4bbd73323b17cd130a, user_id=8cca739a010e4fdfa7d26314242982dd, role_id=0ed49d44afa7491f8cf2b6f367dfcded)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:30.275216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8d48b-2ebc-4d1d-a1ed-070362c6e0d4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:30.334757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.340036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 79/150] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:30 2022] PUT /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a/users/8cca739a010e4fdfa7d26314242982dd/roles/0ed49d44afa7491f8cf2b6f367dfcded => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:30.421484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb152e-9ce1-4039-aab1-5b3a71b3e08c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.421817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb152e-9ce1-4039-aab1-5b3a71b3e08c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.422155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fb152e-9ce1-4039-aab1-5b3a71b3e08c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.423965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 80/151] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:30.429553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.429899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.430281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.472517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:30.473685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:30.502709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56564412-abe1-4212-98d6-28a214c92196 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.548527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:30.559891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.567706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.567706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.567706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.576529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:30.583422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:30.584503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:30.585468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:30.588501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 72/152] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:30.595985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.601345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 81/153] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:30.706407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853ba941-bb90-4c00-a274-8bbd4552409a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.743816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.744375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.744659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.744942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.745654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:30.745999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:30.747419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:30.757600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 73/154] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:30.763321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:30.830008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:30.892213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f41e149-ab88-4ec6-a22e-fa15ed8c9bf6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:30.939381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202ea8c-6f7a-4ac2-85f3-eeda3445e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:30.940030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ea8c-6f7a-4ac2-85f3-eeda3445e4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:30.940369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ea8c-6f7a-4ac2-85f3-eeda3445e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:30.940645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202ea8c-6f7a-4ac2-85f3-eeda3445e4d7 None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:30.941825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ea8c-6f7a-4ac2-85f3-eeda3445e4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:30.943282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202ea8c-6f7a-4ac2-85f3-eeda3445e4d7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:30.945038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 82/155] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:30.951399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.025402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.089663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f837d1f3-915f-48db-838b-311f8daf2e93 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.131413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a6793-8f73-467a-8fdd-c23451ca6b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.132378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a6793-8f73-467a-8fdd-c23451ca6b51 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.132864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a6793-8f73-467a-8fdd-c23451ca6b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.133405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a6793-8f73-467a-8fdd-c23451ca6b51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.192193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0231a1-00a3-45dc-a5d3-359a5071c538 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.192601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0231a1-00a3-45dc-a5d3-359a5071c538 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.192880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0231a1-00a3-45dc-a5d3-359a5071c538 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.194469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 83/156] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:31.214304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb564d-e89d-4fb8-8a8b-e8198e67bda7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.214620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb564d-e89d-4fb8-8a8b-e8198e67bda7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.214850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb564d-e89d-4fb8-8a8b-e8198e67bda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.220526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a6793-8f73-467a-8fdd-c23451ca6b51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.222027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a6793-8f73-467a-8fdd-c23451ca6b51 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.253970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d0cb564d-e89d-4fb8-8a8b-e8198e67bda7 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:31.256996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d0cb564d-e89d-4fb8-8a8b-e8198e67bda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:31.342045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.409946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 84/157] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:31.414308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c75a-7e44-429d-a0cf-d70732733e36 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.414599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c75a-7e44-429d-a0cf-d70732733e36 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.414879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e21c75a-7e44-429d-a0cf-d70732733e36 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.416394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 85/158] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:31.417219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c75a-7e44-429d-a0cf-d70732733e36 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.458298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 74/159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:31.459885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.460376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.460896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.509250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:31.510891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:31.539527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.634565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e21c75a-7e44-429d-a0cf-d70732733e36 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.642652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 75/160] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:31.642652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.693923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.694984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.695509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.695981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.719570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:31.733924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:31.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:31.735866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:31.738416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 86/161] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:31.744476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.806134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.846399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46877623-fbdd-452a-9517-ecd1507f370e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.902747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac71263-8d5d-4416-af93-455b69b4e198 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:31.925280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.925280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.925280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.925280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.931657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.934311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.936077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 76/162] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:31.950272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:31.951044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:31.954532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:31.955482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:31.955972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:31.956485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:31.962697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:31.962697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:31.962697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:31.969099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 87/163] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:31.974764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.118060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f808d1-fe0d-437b-a939-da1bd130d4a9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.167079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.167781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.168136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.168487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] PATH_INFO: `/v3/users/efbdbf875e6a427e9e8f14457c4de416` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.169719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] RBAC: Authorizing `identity:get_user(user_id=efbdbf875e6a427e9e8f14457c4de416)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.171831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.173839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 88/164] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/users/efbdbf875e6a427e9e8f14457c4de416 => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.179276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.208650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fedca1-3a79-471e-a912-b04a429f30ba None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.262387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.265591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.266500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.266609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.266829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.267574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=ffd05526cecb4b7fb2137f53df2eeae7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.267891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.269350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.280610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 77/165] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:31 2022] GET /identity/v3/role_assignments?user.id=ffd05526cecb4b7fb2137f53df2eeae7&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 608 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:32.307895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.329046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8944-9a6e-4b33-aa58-dd69d22dba49 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.340038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.379468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.380107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.380400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.380678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.498041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.499895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:32.544189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc515bb-f583-42c0-ae64-0887b780db17 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.570494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.610146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.610809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.611095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.611373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.632304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:32.637720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.638604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:32.640132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:32.641490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 78/166] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.645753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.712808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 89/167] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:48:32.714015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.721752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:32.753161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dffad7e-1c4b-4317-9cf4-a686b58ae0e9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.794978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.795633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.796469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.796972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.819283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:32.824690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:32.825755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:32.827148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 79/168] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:32.832334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:32.914753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e266e-02bd-480c-86a5-6ee8b44092b7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:32.991664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:32.992290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:32.992573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:32.992848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:32.993526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:32.993861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:32.995277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.006092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.013090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 90/169] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.018685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.032618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c4cb5c-5f86-4a49-bafd-e3d9e19f7784 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.072423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.092459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.093822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.094209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.094494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.098459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.099953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.102021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 80/170] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:32 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.102838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.143868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223b0b8-4a98-4c3f-8c25-87be9eaa1964 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.180355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.180989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.181287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.181572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.182286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:33.182658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.184193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.193456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 91/171] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.194367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.274895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.303456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d80910f-db18-49d6-a39c-1be5cc5ec998 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.319747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1d857-d8a2-40e7-bc95-11b613025afa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.341821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.342449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.343279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.343562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.345073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=efbdbf875e6a427e9e8f14457c4de416) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:33.345395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.346916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.353783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 92/172] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/role_assignments?user.id=efbdbf875e6a427e9e8f14457c4de416&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.359164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.443145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.443823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.444110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.444391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.445564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.447065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.449016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 81/173] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.455721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.524437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef8eb1-767a-4718-9168-4ddebbb87fff None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.567819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.569781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.595100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d454b2-a3b0-4423-9be4-c60f5cbbe6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.596273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d454b2-a3b0-4423-9be4-c60f5cbbe6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.596767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d454b2-a3b0-4423-9be4-c60f5cbbe6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.602325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d454b2-a3b0-4423-9be4-c60f5cbbe6e6 None admin] PATH_INFO: `/v3/users/ffd05526cecb4b7fb2137f53df2eeae7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.603719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d454b2-a3b0-4423-9be4-c60f5cbbe6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=ffd05526cecb4b7fb2137f53df2eeae7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.615181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d454b2-a3b0-4423-9be4-c60f5cbbe6e6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.615181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 93/174] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/users/ffd05526cecb4b7fb2137f53df2eeae7 => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:33.615181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614922b-0235-48b8-a03d-6b32b8cbf2c4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.615181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614922b-0235-48b8-a03d-6b32b8cbf2c4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.615181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614922b-0235-48b8-a03d-6b32b8cbf2c4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.615181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 94/175] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:33.619107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614922b-0235-48b8-a03d-6b32b8cbf2c4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:33.620499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad78d164-54fc-4c77-bfb1-70646539cc55 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.665381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5a1ce-6516-436c-ac23-5744c066d14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.666278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5a1ce-6516-436c-ac23-5744c066d14b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.666879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5a1ce-6516-436c-ac23-5744c066d14b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.667341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5a1ce-6516-436c-ac23-5744c066d14b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.746354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5a1ce-6516-436c-ac23-5744c066d14b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.748146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5a1ce-6516-436c-ac23-5744c066d14b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.794019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614922b-0235-48b8-a03d-6b32b8cbf2c4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:33.837058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.855361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebac576-cf88-4a1e-a9ab-a6800788fcd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:33.856745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebac576-cf88-4a1e-a9ab-a6800788fcd2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.857485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebac576-cf88-4a1e-a9ab-a6800788fcd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.858070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebac576-cf88-4a1e-a9ab-a6800788fcd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.882082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 82/176] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:33.887985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f66a-0809-4d6b-8d0c-e6bfebeb9716 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.888668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f66a-0809-4d6b-8d0c-e6bfebeb9716 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.889126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f66a-0809-4d6b-8d0c-e6bfebeb9716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.911873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b1d3f66a-0809-4d6b-8d0c-e6bfebeb9716 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:33.913063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b1d3f66a-0809-4d6b-8d0c-e6bfebeb9716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:33.945256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:33.956528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebac576-cf88-4a1e-a9ab-a6800788fcd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:33.958623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebac576-cf88-4a1e-a9ab-a6800788fcd2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:33.994564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 83/177] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:33.996279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca4f8-cc8b-47bf-979f-9ae0eb668c02 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:33.996531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca4f8-cc8b-47bf-979f-9ae0eb668c02 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:33.996789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dca4f8-cc8b-47bf-979f-9ae0eb668c02 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:33.998089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 84/178] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:34.003754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca4f8-cc8b-47bf-979f-9ae0eb668c02 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.109881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 95/179] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:34.109881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeacbaa-eb56-4aad-87a8-29fc7f4fb5b0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.109881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeacbaa-eb56-4aad-87a8-29fc7f4fb5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.109881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeacbaa-eb56-4aad-87a8-29fc7f4fb5b0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.109881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 96/180] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:34.113084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a4880-5087-4d77-afd1-a59b8fe9ca31 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.113365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a4880-5087-4d77-afd1-a59b8fe9ca31 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.113639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a4880-5087-4d77-afd1-a59b8fe9ca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.119716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.130633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dca4f8-cc8b-47bf-979f-9ae0eb668c02 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.134321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-894a4880-5087-4d77-afd1-a59b8fe9ca31 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:34.135394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-894a4880-5087-4d77-afd1-a59b8fe9ca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:34.174855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.175499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.175784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.176946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.177543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.179020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.180662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 85/181] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:33 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.186317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.212985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 97/182] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:34.218016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b0a8b-48dc-45ac-bdfb-04cd6f79e718 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.218016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b0a8b-48dc-45ac-bdfb-04cd6f79e718 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.218016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b0a8b-48dc-45ac-bdfb-04cd6f79e718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.237673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-077b0a8b-48dc-45ac-bdfb-04cd6f79e718 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:34.238915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-077b0a8b-48dc-45ac-bdfb-04cd6f79e718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:34.259475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.312471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28c535-6d9a-43df-bca5-13ed253dbf68 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.313858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 98/183] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:34.321592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3614c-3757-4138-9fef-83c2f5f9962e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.322039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3614c-3757-4138-9fef-83c2f5f9962e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.322428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3614c-3757-4138-9fef-83c2f5f9962e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.324855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 99/184] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:34.325651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3614c-3757-4138-9fef-83c2f5f9962e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.356502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.357206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.357492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.357793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] PATH_INFO: `/v3/projects/457097532b824655a88961b47d5ce031` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.359597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] RBAC: Authorizing `identity:get_project(project_id=457097532b824655a88961b47d5ce031)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.363567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.363567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 86/185] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/projects/457097532b824655a88961b47d5ce031 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.363912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.397082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.453416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3614c-3757-4138-9fef-83c2f5f9962e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.471986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd3f62f-4720-4a8f-976c-5567493b7334 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.492700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.493392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.493851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.494225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.495037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] RBAC: Adding query filter params (role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=ffd05526cecb4b7fb2137f53df2eeae7) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.495462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.497113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86c401-1adc-4bad-beaa-ee8518b743b7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.505069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 100/186] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/role_assignments?user.id=ffd05526cecb4b7fb2137f53df2eeae7&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=d0854046eb544eab8092d0a773a97724 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.506333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.506682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.506961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.509090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.509882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.510162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.510443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.523599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:34.528590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:34.529856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:34.531845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:34.533296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:34.533434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 87/187] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:34.533434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.533671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:34.555541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.625183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 101/188] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:34.626203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.644331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.658153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad854c-54c9-4572-853b-7824f27fb414 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.709824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.710456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.710741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.711018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.711758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=457097532b824655a88961b47d5ce031, user.id=ffd05526cecb4b7fb2137f53df2eeae7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.712072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.713531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.720780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 88/189] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/role_assignments?user.id=ffd05526cecb4b7fb2137f53df2eeae7&scope.project.id=457097532b824655a88961b47d5ce031&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.721613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.761234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117e7c31-b2d1-4839-8b7f-8d22d53d1470 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.809539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.810203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.810482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.810755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.812162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:34.812466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.814914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.826048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 102/190] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.830001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.843413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.864957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98e95b-5ffb-4fcd-8b1d-cd33e49a61d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:34.913131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:34.913708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:34.914042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:34.914358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:34.915481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:34.916925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:34.918526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 89/191] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:34.919289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:34.955652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:34.960659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1521ed1e-8b7c-49a5-b8ce-0d03af59aade None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.001311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.001993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.002272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.002542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.013387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:35.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:35.020510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 103/192] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:35.021328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.042200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a0bd-5868-4250-bbd6-e0f9ffa40bab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.078301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.078938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.079226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.079502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] PATH_INFO: `/v3/users/d9b76727b39b43c29405f7ba5c033bcb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.091834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.093711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] RBAC: Authorizing `identity:get_user(user_id=d9b76727b39b43c29405f7ba5c033bcb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.095193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.096868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 90/193] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:34 2022] GET /identity/v3/users/d9b76727b39b43c29405f7ba5c033bcb => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.098098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.123583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892ef648-cf9c-4294-a5c3-ee8431e9ee6b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.158222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.158827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.159111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.159387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.160038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.161485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.188003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.200577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea64f5b-7ce2-4f4f-9a89-5322e09e864b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.235422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.236050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.236337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.236617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.246394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 104/194] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:35 2022] POST /identity/v3/users => generated 297 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:35.247177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:35.247285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:35.253042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:35.254558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 91/195] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:35.255346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.370316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa22baec-4fed-4c52-934a-34fa574dc281 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.374292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c00f6-ac07-4102-a56a-514649662fc2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.407879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.419411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.420262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.420544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.420844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.421416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.421850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.422073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.422211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.422353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.422627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.423948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.439916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 105/196] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.440690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.441823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.510561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.512066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.569360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84601a8f-4e81-44db-b347-711a2db34cd7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.617993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.618628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.618932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.619581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.620441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:35.620758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.622228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.631492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 106/197] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.632329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.645257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 92/198] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:35.649846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.699864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.704071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.778230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b737fb-4708-45ff-9030-53e6be4a8a24 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.789994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da680425-05fd-474c-afd5-e1605f7f19d7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.821633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.822311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.822592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.822865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] PATH_INFO: `/v3/users/d9b76727b39b43c29405f7ba5c033bcb` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.824793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] RBAC: Authorizing `identity:get_user(user_id=d9b76727b39b43c29405f7ba5c033bcb)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.826340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.829110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 107/199] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/users/d9b76727b39b43c29405f7ba5c033bcb => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.830214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.849426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.850581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.851320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.851771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] PATH_INFO: `/v3/users/8cca739a010e4fdfa7d26314242982dd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.853089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] RBAC: Authorizing `identity:get_user(user_id=8cca739a010e4fdfa7d26314242982dd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:35.855247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:35.857521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 93/200] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/users/8cca739a010e4fdfa7d26314242982dd => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:35.862791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:35.944223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5e020-eb4d-4ab5-8b4d-8ef0e4746311 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.970817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:35.987819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0f83c-a25e-435a-9165-f28609a57766 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:35.994659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:35.995619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:35.996101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:35.996578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:35.998468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.001245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.002835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.005335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 108/201] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:36.006549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.032344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.032965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.033241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.033510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.124044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.127016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.149025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e48bc-ebfb-4de9-a4d7-28f2a60048e1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.200428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.201381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.201889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.202357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.261535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.267795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 94/202] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:36.269280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.280189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.282501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.283655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.377907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825983e-7442-4251-b33e-4bc952876808 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.415230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.415843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.416125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.416401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.417158 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=d9b76727b39b43c29405f7ba5c033bcb) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:36.417466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.418946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.425020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 95/203] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/role_assignments?user.id=d9b76727b39b43c29405f7ba5c033bcb&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:36.425846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.517783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 109/204] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:36.524049 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.596241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.623691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c82f88-da83-4a27-99aa-f2637fab90db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.641383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:36.658121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfe65d6-4440-448b-af37-4931e77ebf23 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.673043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.674122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.674612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.675095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.676614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.678995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.686320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 96/205] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:36.688034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.696828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.697473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.697772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.698058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.699187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.700697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.702946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 110/206] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:36.708686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:36.890349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b6314-981c-49a8-899f-bfc29c003e8b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.905808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d1561-ee7a-4b73-b5cf-0cc2cde867e9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:36.959272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.959891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.960284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.960490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.961226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] RBAC: Adding query filter params (role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=d9b76727b39b43c29405f7ba5c033bcb) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:36.961538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.963134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f851e773-50b8-46ce-bcf7-439359b6a40f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:36.971001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 97/207] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/role_assignments?user.id=d9b76727b39b43c29405f7ba5c033bcb&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=d0854046eb544eab8092d0a773a97724 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:36.991321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:36.992025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:36.992257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:36.992540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:36.993311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] RBAC: Adding query filter params (role.id=0ed49d44afa7491f8cf2b6f367dfcded, scope.project.id=27f6325081de4a4bbd73323b17cd130a, user.id=8cca739a010e4fdfa7d26314242982dd) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:36.993627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:36.995223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a740a7-dd17-46a5-b585-d25c54bb9430 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:37.010968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 111/208] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:36 2022] GET /identity/v3/role_assignments?user.id=8cca739a010e4fdfa7d26314242982dd&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=0ed49d44afa7491f8cf2b6f367dfcded => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:37.463217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2107f-2f0e-46c1-8f6d-a9140678e4a7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.464129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2107f-2f0e-46c1-8f6d-a9140678e4a7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.464409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b2107f-2f0e-46c1-8f6d-a9140678e4a7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.467903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 98/209] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:37.476962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.477249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.477531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.504700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:37.506316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:37.521768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.586768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad187f-2498-4f01-8bc2-ac8ca59c7592 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.587121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad187f-2498-4f01-8bc2-ac8ca59c7592 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.587329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad187f-2498-4f01-8bc2-ac8ca59c7592 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.588765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 99/210] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:37.591503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 112/211] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:37.594689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:37.594987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:37.595252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:37.612962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:37.619507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:37.620713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:37.699662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 100/212] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:37.883301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:37.975097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.000258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891256e-1c9d-49af-9cf4-b619dec32701 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.044638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.046174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.047013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.047498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.062430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:38.068585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:38.069526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:38.070373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 113/213] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:38.077450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.089301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.097436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba95312-f296-4cf3-85b3-e6f039580168 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.135586 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.136159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.136446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.136724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.138017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.140032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.141976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 101/214] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:38.147868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.152941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.209521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8f20a-dad1-4033-969e-69450a5a8518 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.246809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.247418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.247702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.248058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.248708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:38.249105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.250730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.259623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 114/215] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:38.265653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.266609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8736590-e2f8-44cc-9531-ecc9757e9824 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.304376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9a8ac-a0ca-44d9-be67-9686fca309a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.305406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9a8ac-a0ca-44d9-be67-9686fca309a1 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.305938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9a8ac-a0ca-44d9-be67-9686fca309a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.306458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9a8ac-a0ca-44d9-be67-9686fca309a1 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.307339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9a8ac-a0ca-44d9-be67-9686fca309a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.309033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf9a8ac-a0ca-44d9-be67-9686fca309a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.340162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.396308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c5ada-90cc-4fb5-bc28-5b3fa6b1ef02 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.432987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.433625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.433944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.434222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] PATH_INFO: `/v3/users/3c4314e3a7e042deb6080d4a2ad74bcd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.448327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] RBAC: Authorizing `identity:get_user(user_id=3c4314e3a7e042deb6080d4a2ad74bcd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.449821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.451478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 115/216] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/users/3c4314e3a7e042deb6080d4a2ad74bcd => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:38.457440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.460707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 102/217] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:38 2022] POST /identity/v3/users => generated 297 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:48:38.588650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487fca4-d096-4299-a311-718c912ad812 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:38.623317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.631753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda30457-b0a5-4015-935c-fe37e2f364e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:38.632378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda30457-b0a5-4015-935c-fe37e2f364e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.632689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda30457-b0a5-4015-935c-fe37e2f364e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.632972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda30457-b0a5-4015-935c-fe37e2f364e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.661656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cfed4-9e25-47b3-a092-ab1e16866efa None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.662179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cfed4-9e25-47b3-a092-ab1e16866efa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.662179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729cfed4-9e25-47b3-a092-ab1e16866efa None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.663789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 103/218] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:38.669496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:38.669935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:38.670280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:38.694859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:38.696043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:38.719342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda30457-b0a5-4015-935c-fe37e2f364e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:38.720863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda30457-b0a5-4015-935c-fe37e2f364e3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:38.784878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:38.805236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 104/219] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:38.858407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 116/220] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:48:38.864850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:38.975141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37afeec-c340-4204-974b-3b31aeec83d1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.009772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20373-0251-4e39-b870-6360967b5f74 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.010306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20373-0251-4e39-b870-6360967b5f74 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.010306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf20373-0251-4e39-b870-6360967b5f74 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.011934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 117/221] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:39.017621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.018641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.018971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.019264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.019533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.019632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.020113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.022856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.024321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.026030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 105/222] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:38 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.038880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.047332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:39.048532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:39.102218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.135741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.149553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 118/223] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:39.150521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.193801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fb7dd-0983-4c98-b03e-6cc6bc9f178c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.233970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.235682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.235967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.236240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.236980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=3c4314e3a7e042deb6080d4a2ad74bcd) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.237288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.238835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.249767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 106/224] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/role_assignments?user.id=3c4314e3a7e042deb6080d4a2ad74bcd&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.278080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760a9755-483e-41b0-8dee-2941abc0d6ae None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.318572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.319232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.319510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.319784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.334388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:39.339641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.340568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:39.341608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 119/225] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.348575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.515003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4791b1c0-f54f-40f9-a00c-cecaf3ae0c03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.540647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.558866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.560037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.560679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.562027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.562027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.562724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:39.563033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.564505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.575286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 107/226] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.581098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.601301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.699616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8497aa-35c2-4830-8516-2c6a82e0b7b6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.754127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03be55-964c-4dfe-b416-d6617d88e564 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:39.767925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.768626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.768911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.769213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] PATH_INFO: `/v3/users/d9b76727b39b43c29405f7ba5c033bcb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.774551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] RBAC: Authorizing `identity:get_user(user_id=d9b76727b39b43c29405f7ba5c033bcb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:39.776046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:39.778087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 108/227] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/users/d9b76727b39b43c29405f7ba5c033bcb => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:39.787056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.800353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:39.801377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:39.801880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:39.802369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:39.821620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:39.828660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:39.829516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:39.830323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:39.834529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 120/228] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:39.845637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:39.918056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:39.960845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.003640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3902a646-dc54-4921-9bd5-f4aa43413f81 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.015851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92b2fa7-8e2e-4519-a078-0b44b165326d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.073863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.074493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.074774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.075048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.075850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:40.076244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.077857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.080503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.081975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.082408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.082856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.087227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.088108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.089831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 109/229] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.094245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 121/230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.102731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.102883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.297413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3f4ef-2775-4c86-81d5-a3c7221e7b14 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.300916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef97483-9229-45c2-bc46-a1b57bd0818d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.338977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.366051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.366698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.366977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.367252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] PATH_INFO: `/v3/users/d9b76727b39b43c29405f7ba5c033bcb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.370138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] RBAC: Authorizing `identity:get_user(user_id=d9b76727b39b43c29405f7ba5c033bcb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.371899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.372562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.372841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.373116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/d9b76727b39b43c29405f7ba5c033bcb/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.375048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.382542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] RBAC: Authorizing `identity:create_grant(project_id=c2256eaae8634eaaabf64f763a14d6b7, user_id=d9b76727b39b43c29405f7ba5c033bcb, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.384037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 110/231] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/users/d9b76727b39b43c29405f7ba5c033bcb => generated 283 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:40.384414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.388192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.454694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:40.459189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 122/232] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:40 2022] PUT /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/d9b76727b39b43c29405f7ba5c033bcb/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:40.540812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf70b7-f65c-463c-a9b8-bb1fa6495b74 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.594911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d44e92-ba44-4240-99f1-33bd5fbb1c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.595503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44e92-ba44-4240-99f1-33bd5fbb1c58 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.595788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44e92-ba44-4240-99f1-33bd5fbb1c58 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.596065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d44e92-ba44-4240-99f1-33bd5fbb1c58 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.597301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d44e92-ba44-4240-99f1-33bd5fbb1c58 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.598797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d44e92-ba44-4240-99f1-33bd5fbb1c58 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.601916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 111/233] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:40 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:40.607360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:40.784382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63437-afbb-4387-bc2f-cef21524f9db None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:40.839557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58532ff1-1c94-4ce2-957f-e70e0c65b64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:40.840224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58532ff1-1c94-4ce2-957f-e70e0c65b64d None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.840503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58532ff1-1c94-4ce2-957f-e70e0c65b64d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.840774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58532ff1-1c94-4ce2-957f-e70e0c65b64d None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/d9b76727b39b43c29405f7ba5c033bcb/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.847612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58532ff1-1c94-4ce2-957f-e70e0c65b64d None admin] RBAC: Authorizing `identity:create_grant(project_id=c2256eaae8634eaaabf64f763a14d6b7, user_id=d9b76727b39b43c29405f7ba5c033bcb, role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:40.849042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58532ff1-1c94-4ce2-957f-e70e0c65b64d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:40.932771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 123/234] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:40 2022] PUT /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/d9b76727b39b43c29405f7ba5c033bcb/roles/d0854046eb544eab8092d0a773a97724 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:40.974925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae2a82-ad4f-46f2-affe-bf462479e0cc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.975482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae2a82-ad4f-46f2-affe-bf462479e0cc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.975482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ae2a82-ad4f-46f2-affe-bf462479e0cc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:40.976866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 112/235] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:40.988626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45910264-8ef8-4134-896a-1571328d7a99 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:40.988938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45910264-8ef8-4134-896a-1571328d7a99 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:40.989183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45910264-8ef8-4134-896a-1571328d7a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.018998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-45910264-8ef8-4134-896a-1571328d7a99 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:41.020215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-45910264-8ef8-4134-896a-1571328d7a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:41.114911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.134066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 124/236] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:41.135476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07cde-4fd6-475e-b26a-ac7209cc0ae4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.139827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07cde-4fd6-475e-b26a-ac7209cc0ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.139827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07cde-4fd6-475e-b26a-ac7209cc0ae4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.139827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 113/237] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:41.143410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e6d2-6f4a-4804-974c-0af909fbcf17 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.143699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e6d2-6f4a-4804-974c-0af909fbcf17 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.143972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c8e6d2-6f4a-4804-974c-0af909fbcf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.166736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-02c8e6d2-6f4a-4804-974c-0af909fbcf17 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:41.168209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-02c8e6d2-6f4a-4804-974c-0af909fbcf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:41.247910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 125/238] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:41.584106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07cde-4fd6-475e-b26a-ac7209cc0ae4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.596276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3abf8-d7bd-49fb-aae8-8e1171487d48 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.596576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3abf8-d7bd-49fb-aae8-8e1171487d48 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.596849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3abf8-d7bd-49fb-aae8-8e1171487d48 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.598254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 126/239] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:41.605205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016f252-82bb-4296-8549-59865faee875 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.605794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016f252-82bb-4296-8549-59865faee875 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.606297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016f252-82bb-4296-8549-59865faee875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.628764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4016f252-82bb-4296-8549-59865faee875 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:41.630314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4016f252-82bb-4296-8549-59865faee875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:41.698111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.753837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 127/240] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:41.753837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016f252-82bb-4296-8549-59865faee875 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.760429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07cde-4fd6-475e-b26a-ac7209cc0ae4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.810168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:41.830437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.831495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.831996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.832470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.852490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:41.857719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:41.858697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:41.859474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:41.867767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 114/241] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:41.869144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016f252-82bb-4296-8549-59865faee875 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:41.874027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:41.908555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:41.909499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:41.909637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:41.909870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:41.911100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:41.918747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:41.918747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 128/242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:41.919906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.025748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc19ca-adbb-4c0d-851c-cef6b6b039c1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.054370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.062685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.063552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.064006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.065285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.067019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.077199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a14de-ae19-421e-84d9-11e4614987a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.114122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.121109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.121723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.122032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.122312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.123045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:42.123898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.125522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.134521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 115/243] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:42.139845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.188789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 129/244] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:41 2022] POST /identity/v3/users => generated 293 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:48:42.189979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.298430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7b6ab-fb15-450e-8250-a5051797e669 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.375656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.376325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.376614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.376964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] PATH_INFO: `/v3/users/be464955f93045eca0f27b60ac46d032` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.396278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] RBAC: Authorizing `identity:get_user(user_id=be464955f93045eca0f27b60ac46d032)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.398647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.401768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 116/245] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/users/be464955f93045eca0f27b60ac46d032 => generated 297 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:42.406490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.409419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f1325-8c40-4b82-b142-62b891977a97 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.460211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.460405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.466177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.466782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.467284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.483495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:42.491554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:42.492543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:42.493629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 130/246] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:42.497782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.546694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.559892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9545-3b77-42c4-b2a7-b6a89c97a322 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.598194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72fb397-5a46-4ffd-9dbe-04f97463fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.598857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb397-5a46-4ffd-9dbe-04f97463fe73 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.599144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb397-5a46-4ffd-9dbe-04f97463fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.599430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72fb397-5a46-4ffd-9dbe-04f97463fe73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.636005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631da17b-efac-4ed4-adf0-bc993e45b163 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.687377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.688272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.688560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.689460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.690199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:42.690658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.691646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb397-5a46-4ffd-9dbe-04f97463fe73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.692215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.693336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72fb397-5a46-4ffd-9dbe-04f97463fe73 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.704140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 131/247] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:42.709910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.746551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.839581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 117/248] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:42.841055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8397d-d933-42bf-a0ed-d4cf703b80b5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.841368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8397d-d933-42bf-a0ed-d4cf703b80b5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.841642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf8397d-d933-42bf-a0ed-d4cf703b80b5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.843666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 118/249] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:42.846560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:42.849582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.849905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.850183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.853860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d46f36-5027-4caf-be63-d8c3fc3ec80e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:42.870858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:42.872059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:42.899857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:42.900517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:42.900798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:42.901072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] PATH_INFO: `/v3/users/3c4314e3a7e042deb6080d4a2ad74bcd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:42.903073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] RBAC: Authorizing `identity:get_user(user_id=3c4314e3a7e042deb6080d4a2ad74bcd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:42.904540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:42.906609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 132/250] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/users/3c4314e3a7e042deb6080d4a2ad74bcd => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:42.907413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:42.974076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 119/251] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:42.975354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.036755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0643e1c8-62fb-4830-9cad-3bb9b4c327f8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.048554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.078330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.078980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.079265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.079545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.080809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.082613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.084190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 133/252] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:43.087968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa2ed8-df75-4d25-9191-fc4795843b70 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.090119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.125290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.127448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.128347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.128890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.129497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.130919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.132614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.135086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 120/253] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:42 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:43.147166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.267743 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f625ff5-c5b0-4205-8c1f-952f36d8ffe8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.319159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.319846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.320130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.323419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.323530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a283b7d-fac2-4e04-afe0-25e315444126 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.323622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] RBAC: Adding query filter params (role.id=0ed49d44afa7491f8cf2b6f367dfcded, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=be464955f93045eca0f27b60ac46d032) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:43.323622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.323622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff160b65-c168-42b6-b620-f7f301e393e1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.330365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 134/254] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/role_assignments?user.id=be464955f93045eca0f27b60ac46d032&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=0ed49d44afa7491f8cf2b6f367dfcded => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:43.332031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc0986-f127-4bc1-989d-068c7274ebfc None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.332031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc0986-f127-4bc1-989d-068c7274ebfc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.338014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc0986-f127-4bc1-989d-068c7274ebfc None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.339834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 135/255] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:43.348365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.348531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.348783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.408268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.414817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.415034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:43.415574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.416072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/3c4314e3a7e042deb6080d4a2ad74bcd/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.416280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:43.418807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2256eaae8634eaaabf64f763a14d6b7, user_id=3c4314e3a7e042deb6080d4a2ad74bcd, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.421199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.430300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.523204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 136/256] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:43.528187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.539059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 121/257] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:43 2022] PUT /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/3c4314e3a7e042deb6080d4a2ad74bcd/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:43.579277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:43.691004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c768dc4-d8d4-4c4d-a1b3-07491107128c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.739497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.740516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.741002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.741503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.753106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:43.758378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:43.759266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:48:43.760060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:43.762373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 137/258] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:43.769420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:43.895464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407fd662-9ce3-427e-ab0e-0dbd99854353 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:43.941150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:43.942202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:43.942686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:43.943180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:43.944067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:43.944571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:43.946217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:43.962431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 122/259] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:43.968142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.072449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.090090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7fd55-b9fc-4434-9787-ddbdb2c7555f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.133259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.134292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.134800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.135284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] PATH_INFO: `/v3/users/d9b76727b39b43c29405f7ba5c033bcb` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.141064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] RBAC: Authorizing `identity:get_user(user_id=d9b76727b39b43c29405f7ba5c033bcb)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.142868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.144799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 138/260] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:43 2022] GET /identity/v3/users/d9b76727b39b43c29405f7ba5c033bcb => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:44.150706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.169247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.270625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.364087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c934785-3d66-4dfc-8ecc-f9cee64053a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.417683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609c4e5-ef30-421d-b75a-570eb909d7ff None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.459336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.459959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.460238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.460512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.484602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.485241 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.485525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.485833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.510967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:44.516569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:44.517437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:44.518265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:44.522453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 139/261] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:48:44.528114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.563584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.641930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.643416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.741264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c455af03-035a-4b7a-9e68-253fc0f53fe8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:44.761981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:44.806286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:44.806903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:44.807180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:44.807455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:44.808222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:44.808500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:44.809933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:44.826151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 140/262] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:44.834048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.841075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 123/263] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 690 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:44.847130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:44.982204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.019406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5c1d39-1c2c-4dec-8722-f8b112172d52 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.031356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce28200-18b5-4c0b-90d7-e1091d4bac71 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.082553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.083107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.083386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.083657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] PATH_INFO: `/v3/users/d9b76727b39b43c29405f7ba5c033bcb` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.084951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] RBAC: Authorizing `identity:get_user(user_id=d9b76727b39b43c29405f7ba5c033bcb)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.086444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.088029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 141/264] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/users/d9b76727b39b43c29405f7ba5c033bcb => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.094034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.094694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.094970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.095246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.096576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.097518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.098976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.100596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 124/265] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:44 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.110953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.153852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.271758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b564fbd-572b-46d2-a80a-e9ada5fafdb6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.296924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f1e08-f4dc-4029-8d8e-014782024dc5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.347198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.347766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.348056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.348337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.349080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=d9b76727b39b43c29405f7ba5c033bcb) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:45.349424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.350907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20973436-8ebf-4f25-aa0d-720dae39cec4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.363731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 125/266] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/role_assignments?user.id=d9b76727b39b43c29405f7ba5c033bcb&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:45.369329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.370405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.370905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.371380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.396147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.507675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c717e-00c1-42e5-9f34-769f357c1113 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.508023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c717e-00c1-42e5-9f34-769f357c1113 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.508233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64c717e-00c1-42e5-9f34-769f357c1113 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.509590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 126/267] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:45.515898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055d89a-c7f6-43c3-8ca8-48f896a7e017 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.516192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055d89a-c7f6-43c3-8ca8-48f896a7e017 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.516466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055d89a-c7f6-43c3-8ca8-48f896a7e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.530011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.532316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.566912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a055d89a-c7f6-43c3-8ca8-48f896a7e017 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:45.568131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a055d89a-c7f6-43c3-8ca8-48f896a7e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:45.703702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 127/268] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:45.704946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8da6-7271-4a61-a058-ab78b295bf5c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.705234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8da6-7271-4a61-a058-ab78b295bf5c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.705514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6a8da6-7271-4a61-a058-ab78b295bf5c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.706811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 128/269] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:45.711720 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 142/270] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:45.712368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.712698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.713015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.718341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:45.734400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:45.735613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:45.759799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.777426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:45.845607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a4b930-837a-4e7f-ba12-13afe0347ae7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:45.891227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:45.892398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:45.892860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:45.893282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:45.894783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:45.896496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:45.898225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 129/271] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:45.898607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 143/272] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:45.905192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.015907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cb2fff-e511-4312-b079-c46aa633f65b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.040643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.059181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.060172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.060641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.061702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.062653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] RBAC: Adding query filter params (role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=d9b76727b39b43c29405f7ba5c033bcb) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.063222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.064875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096cb90-d65c-4aea-8cdd-b9a6bc44b78a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.072011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 130/273] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:45 2022] GET /identity/v3/role_assignments?user.id=d9b76727b39b43c29405f7ba5c033bcb&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=d0854046eb544eab8092d0a773a97724 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:46.082827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568783eb-b91e-4b8e-9fac-332b2888d398 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.082827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568783eb-b91e-4b8e-9fac-332b2888d398 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.083061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568783eb-b91e-4b8e-9fac-332b2888d398 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.084641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 131/274] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:46.089882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.090142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.090579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.124881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:46.128255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:46.145427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.202486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0c9b3-54ee-4c6c-9639-ab097f9fefba None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.219202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.245254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.246513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.247004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.247480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.259581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:46.267014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:46.268050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:46.268954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:46.273867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 144/275] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:46.274012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 132/276] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:46.278453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.335620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.393972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f87f0be-1c26-46f7-8650-3f42a72f9e5f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.436009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.437469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.437960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.438416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.439500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.439998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.442290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.448777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2d8fce-d6d5-41d0-bfc4-43d781ce11d9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.452590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 145/277] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.461455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.504605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.513885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.514725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.515475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.520526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.520526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.520526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.521525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 133/278] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:46.527988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.543910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.594843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5c8544-f6c9-4550-b34a-e49c4af2e59a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.630401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd55bd-6626-4a49-87a1-f62d72fad39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.631032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd55bd-6626-4a49-87a1-f62d72fad39b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.631312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd55bd-6626-4a49-87a1-f62d72fad39b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.631585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd55bd-6626-4a49-87a1-f62d72fad39b None admin] PATH_INFO: `/v3/users/91453f6e66e8402d919b2b44e638ff74` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.645591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd55bd-6626-4a49-87a1-f62d72fad39b None admin] RBAC: Authorizing `identity:get_user(user_id=91453f6e66e8402d919b2b44e638ff74)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.646757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd753b-38cf-4861-8c29-0d5376922228 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.647134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd55bd-6626-4a49-87a1-f62d72fad39b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.648862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 146/279] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/users/91453f6e66e8402d919b2b44e638ff74 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:46.650301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34b6a-b8c3-45b5-bfca-3f8a115095ec None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.650587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34b6a-b8c3-45b5-bfca-3f8a115095ec None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.650857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34b6a-b8c3-45b5-bfca-3f8a115095ec None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.652088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 147/280] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:46.652788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f34b6a-b8c3-45b5-bfca-3f8a115095ec None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.690324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.690949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.691234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.691517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.692184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.693639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.750666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.760751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f34b6a-b8c3-45b5-bfca-3f8a115095ec None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.797641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 134/281] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:48:46 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:46.798561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.799545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b27d39-ebab-4ca6-966a-53c2cfa5e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.801013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27d39-ebab-4ca6-966a-53c2cfa5e253 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.801845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27d39-ebab-4ca6-966a-53c2cfa5e253 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.803499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b27d39-ebab-4ca6-966a-53c2cfa5e253 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.825533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b27d39-ebab-4ca6-966a-53c2cfa5e253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-99b27d39-ebab-4ca6-966a-53c2cfa5e253 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:46.834647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:46.835653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:46.836468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 148/282] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:46.842122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.842122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.842122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.873041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:46.875034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:46.882039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:46.928702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02367964-36af-4da6-a4c7-d9cfa55a25ea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:46.956512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 149/283] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:46.957375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:46.975593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76077138-e631-4952-ac41-82832c87d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:46.976217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76077138-e631-4952-ac41-82832c87d930 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:46.976507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76077138-e631-4952-ac41-82832c87d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:46.976784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76077138-e631-4952-ac41-82832c87d930 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:46.977519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76077138-e631-4952-ac41-82832c87d930 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, user.id=91453f6e66e8402d919b2b44e638ff74, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:46.977861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76077138-e631-4952-ac41-82832c87d930 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:46.979306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76077138-e631-4952-ac41-82832c87d930 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:46.985919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 135/284] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/role_assignments?user.id=91453f6e66e8402d919b2b44e638ff74&scope.system=all&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.068387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.083361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b907a9-6698-4afb-bb60-705af6d57f7f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.123604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.124248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.124533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.124811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.125503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:47.125881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.127289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.136944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 150/285] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.143427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76077138-e631-4952-ac41-82832c87d930 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.230483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.271680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76077138-e631-4952-ac41-82832c87d930 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.306184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.312074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.312849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.313135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.325813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] PATH_INFO: `/v3/users/be464955f93045eca0f27b60ac46d032` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.325813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] RBAC: Authorizing `identity:get_user(user_id=be464955f93045eca0f27b60ac46d032)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.325813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.325813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 136/286] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/users/be464955f93045eca0f27b60ac46d032 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.325813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.354924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0619d5-c75a-46bf-96a2-16651524409b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.391585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.392579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.393055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.393709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.406637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:47.412812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:47.413768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:47.414585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:47.417488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 151/287] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:47.424304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.437147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.476292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dedea6-d266-4daa-8d18-b902fa0a0f19 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.520706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.521369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.521651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.521947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.525950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.527448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.534756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 137/288] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:47.541021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.634221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.659246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a0e35-8144-4cec-8a92-c1ae66e98a59 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.672992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118ac96f-fee2-45f8-b34b-e5dc344c04a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.717522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.718443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.718734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.719014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.719707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:47.720017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.725757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.726944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.727878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.728365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.728846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/be464955f93045eca0f27b60ac46d032/roles/0ed49d44afa7491f8cf2b6f367dfcded` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.735528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=c2256eaae8634eaaabf64f763a14d6b7, user_id=be464955f93045eca0f27b60ac46d032, role_id=0ed49d44afa7491f8cf2b6f367dfcded)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.738038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.738261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 152/289] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:47.746388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:47.823879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 138/290] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:48:47 2022] PUT /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/users/be464955f93045eca0f27b60ac46d032/roles/0ed49d44afa7491f8cf2b6f367dfcded => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:48:47.833151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:47.888052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e5710-4751-42d2-bdfb-29f72ed1bc0a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:47.942485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:47.943138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:47.943424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:47.943705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] PATH_INFO: `/v3/users/3c4314e3a7e042deb6080d4a2ad74bcd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:47.949058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] RBAC: Authorizing `identity:get_user(user_id=3c4314e3a7e042deb6080d4a2ad74bcd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:47.950776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:47.952544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 153/291] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/users/3c4314e3a7e042deb6080d4a2ad74bcd => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:47.958624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.085874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31faa25-69e3-4e04-975a-fae73cb53eaf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.122714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.132911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.133589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.133913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.134197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.218736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.220199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.351869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 139/292] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:48.357901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.479132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671f31a-26e4-457f-97b7-23981ea2f4fc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.516016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674bd4c9-a163-482d-b9da-d03043586f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.516667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd4c9-a163-482d-b9da-d03043586f93 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.516960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd4c9-a163-482d-b9da-d03043586f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.517244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674bd4c9-a163-482d-b9da-d03043586f93 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.518482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd4c9-a163-482d-b9da-d03043586f93 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.520231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674bd4c9-a163-482d-b9da-d03043586f93 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.522079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 154/293] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:48.527863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:48.556857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:48.658818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be071b26-0dc7-4a43-9704-b42f0873469c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:48.723042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:48.724049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:48.724528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:48.725000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:48.725974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=3c4314e3a7e042deb6080d4a2ad74bcd) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:48.726501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:48.728125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a88d0-4428-43c0-81f5-c32eac01bdb7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:48.738450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 140/294] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:48 2022] GET /identity/v3/role_assignments?user.id=3c4314e3a7e042deb6080d4a2ad74bcd&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:49.291884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff5d9c-1321-450e-af80-182f665c3be2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.292392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff5d9c-1321-450e-af80-182f665c3be2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.292500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bff5d9c-1321-450e-af80-182f665c3be2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.294160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 155/295] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:49.299828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7069-5631-415f-adae-6010ecff3b6d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.300127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7069-5631-415f-adae-6010ecff3b6d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.300412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb7069-5631-415f-adae-6010ecff3b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.324564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d4eb7069-5631-415f-adae-6010ecff3b6d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:49.326799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a89a-4e44-4565-affa-f25a6abb4500 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.327285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a89a-4e44-4565-affa-f25a6abb4500 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.327741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a89a-4e44-4565-affa-f25a6abb4500 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.329035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d4eb7069-5631-415f-adae-6010ecff3b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.329483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 156/296] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:49.341983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c97e1c-19c0-4753-969e-72ca4375faad None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.342634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c97e1c-19c0-4753-969e-72ca4375faad None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.343114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c97e1c-19c0-4753-969e-72ca4375faad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.359831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.380274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-26c97e1c-19c0-4753-969e-72ca4375faad None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:49.381873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-26c97e1c-19c0-4753-969e-72ca4375faad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.407979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 141/297] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:49.463863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 157/298] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:49.570295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109b31b-2f8b-4b6c-9ff0-0deb8e2f3221 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.570917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109b31b-2f8b-4b6c-9ff0-0deb8e2f3221 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.571388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109b31b-2f8b-4b6c-9ff0-0deb8e2f3221 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.573165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 142/299] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:49.578319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.578831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.579306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.598862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:49.600300 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:49.674762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 158/300] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:49.681076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109b31b-2f8b-4b6c-9ff0-0deb8e2f3221 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.705096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.746800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.789718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109b31b-2f8b-4b6c-9ff0-0deb8e2f3221 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.816765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.825358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.825962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.826334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.827167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.838132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:49.843390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:49.844286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:49.845118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 143/301] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:49.850330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.868959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a616bc7-b7a8-4f2f-af0d-a48d67540024 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:49.910045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:49.911164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:49.911922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:49.912675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:49.926758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:49.932428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:49.933332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:49.934151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:49.937193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 159/302] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:49.943113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:49.957427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:49.979607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffa8cb-dafe-42ca-9853-0be964bb850e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.018632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.019734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.020205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.020665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.021515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.022053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.023530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.032564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 144/303] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.033439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.063773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1ae00-a470-4d70-b803-8ae5306478c8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.090208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.103137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.108903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.108903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.108903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.108903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.108903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.108903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.116693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 160/304] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:50.117752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.159061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e612a5a-f980-4dfb-8938-3c812b5fe28e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.196247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.196885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.197165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.197439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.198587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.200026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.202103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 145/305] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.206628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.220870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.231679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c0bd6-0ecb-4497-9049-adda0d99b60e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.273521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.274475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.274904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.275333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] PATH_INFO: `/v3/users/91453f6e66e8402d919b2b44e638ff74` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.276944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] RBAC: Authorizing `identity:get_user(user_id=91453f6e66e8402d919b2b44e638ff74)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.280043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.282708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 161/306] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/users/91453f6e66e8402d919b2b44e638ff74 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.283632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.329331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcba57f-c1e5-4abc-bf20-e053dd52e38e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.354562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.366359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.366999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.367282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.367563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] PATH_INFO: `/v3/users/24cd5f9ff29e427f9bc91d175d60fd85` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.381309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] RBAC: Authorizing `identity:get_user(user_id=24cd5f9ff29e427f9bc91d175d60fd85)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.383299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.384524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 146/307] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/users/24cd5f9ff29e427f9bc91d175d60fd85 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.385284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.393182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2ee0a-779f-4672-a194-2283032df398 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.428545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27cdd0c-7b52-49c8-862c-ec08ca06f1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.435319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cdd0c-7b52-49c8-862c-ec08ca06f1a6 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.435319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cdd0c-7b52-49c8-862c-ec08ca06f1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.435319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27cdd0c-7b52-49c8-862c-ec08ca06f1a6 None admin] PATH_INFO: `/v3/groups` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.435319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cdd0c-7b52-49c8-862c-ec08ca06f1a6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.435319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27cdd0c-7b52-49c8-862c-ec08ca06f1a6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.457655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.491535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa090fa1-3aa3-4ba3-9713-75ad7d0bc133 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.525015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff46588-09bf-4f0c-93b9-2499939a5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.525666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46588-09bf-4f0c-93b9-2499939a5681 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.525978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46588-09bf-4f0c-93b9-2499939a5681 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.526261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46588-09bf-4f0c-93b9-2499939a5681 None admin] PATH_INFO: `/v3/system/users/91453f6e66e8402d919b2b44e638ff74/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.527659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff46588-09bf-4f0c-93b9-2499939a5681 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=91453f6e66e8402d919b2b44e638ff74, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.529053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff46588-09bf-4f0c-93b9-2499939a5681 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.549770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 162/308] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Aug 30 00:48:50 2022] POST /identity/v3/groups => generated 229 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:50.553943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fe9f-dfb3-4cd8-8598-348d9dcf8100 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.553943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fe9f-dfb3-4cd8-8598-348d9dcf8100 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.553943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fe9f-dfb3-4cd8-8598-348d9dcf8100 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.553943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 163/309] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:50.553943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352fe9f-dfb3-4cd8-8598-348d9dcf8100 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:50.675455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 147/310] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Aug 30 00:48:50 2022] PUT /identity/v3/system/users/91453f6e66e8402d919b2b44e638ff74/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:50.676778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.677064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.677341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.697978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352fe9f-dfb3-4cd8-8598-348d9dcf8100 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:50.702489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:50.703867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:50.718958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:50.760001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:50.760237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:50.760655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:50.760983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:50.761919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] RBAC: Adding query filter params (role.id=0ed49d44afa7491f8cf2b6f367dfcded, user.id=24cd5f9ff29e427f9bc91d175d60fd85, scope.system=all) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:50.762396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:50.765209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:50.771475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 164/311] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:50 2022] GET /identity/v3/role_assignments?user.id=24cd5f9ff29e427f9bc91d175d60fd85&scope.system=all&role.id=0ed49d44afa7491f8cf2b6f367dfcded => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:50.814203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 148/312] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:51.083047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.197933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ef130-2134-473e-9204-ceb3b275e7bb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.234163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.234814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.235101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.235430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.248048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:51.253899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:51.254841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:51.255681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:51.258343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 165/313] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:51.264472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.332451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.374515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d5b85-db01-405c-bf1a-dbc7309a31ee None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.411760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.412805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.413346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.413890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.414824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:51.415335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.416947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.427093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 149/314] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:51.432839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.471808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.540925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a143ca4-f751-4965-87ac-8b0711989a4a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.578117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.578757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.579043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.579320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] PATH_INFO: `/v3/users/be464955f93045eca0f27b60ac46d032` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.580468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] RBAC: Authorizing `identity:get_user(user_id=be464955f93045eca0f27b60ac46d032)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.581963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.583544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 166/315] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/users/be464955f93045eca0f27b60ac46d032 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:51.589292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:51.707173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa67b35-89de-41bd-b4ee-33c9083b80d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:51.732976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:51.746443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:51.747102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:51.747526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:51.747812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:51.826224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:51.827917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:51.935254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 150/316] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:51.943361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.056280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa18f709-63ee-44c9-b879-55807a4f729d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.086578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.093094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8938f-508b-49bd-8f63-2e9dc5f02a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.093756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8938f-508b-49bd-8f63-2e9dc5f02a91 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.094059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8938f-508b-49bd-8f63-2e9dc5f02a91 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.094337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8938f-508b-49bd-8f63-2e9dc5f02a91 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.095476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8938f-508b-49bd-8f63-2e9dc5f02a91 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.096997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8938f-508b-49bd-8f63-2e9dc5f02a91 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.098954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 167/317] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:51 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.105152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:52.138443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.212864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ed0d6-6d13-4ac1-8613-7111fe3a02ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:52.250412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:52.251073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.251366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.251648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.252383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] RBAC: Adding query filter params (role.id=0ed49d44afa7491f8cf2b6f367dfcded, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7, user.id=be464955f93045eca0f27b60ac46d032) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:52.252697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:52.254183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938547a-cb28-4e02-a484-8cbf24e5b7a8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:52.260639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 151/318] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:48:52 2022] GET /identity/v3/role_assignments?user.id=be464955f93045eca0f27b60ac46d032&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=0ed49d44afa7491f8cf2b6f367dfcded => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:52.662565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8215e1-ad4a-422d-a0b8-2d73328fd3c2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.663296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8215e1-ad4a-422d-a0b8-2d73328fd3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.663296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8215e1-ad4a-422d-a0b8-2d73328fd3c2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.664420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 168/319] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:52.669948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fe4ae-beff-41f4-b570-12a52be9fc0e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.670574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fe4ae-beff-41f4-b570-12a52be9fc0e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.670574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fe4ae-beff-41f4-b570-12a52be9fc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.690209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3c0fe4ae-beff-41f4-b570-12a52be9fc0e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:52.691190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3c0fe4ae-beff-41f4-b570-12a52be9fc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.691623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4c02b-b264-45e8-8428-ab28aecd69c6 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.691917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4c02b-b264-45e8-8428-ab28aecd69c6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.692186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c4c02b-b264-45e8-8428-ab28aecd69c6 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.693683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 169/320] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:52.698473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.698761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.699036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.719631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:52.720838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.772852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 152/321] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:52.775296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5f7c3-2028-472c-83d4-f539ca983c06 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.775656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5f7c3-2028-472c-83d4-f539ca983c06 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.775866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5f7c3-2028-472c-83d4-f539ca983c06 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.777115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 153/322] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:52.782369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:52.782920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:52.783411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:52.798840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 170/323] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:52.802785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:52.804224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:52.819519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:52.874946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 154/324] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:53.040854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.084612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.157566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e363dbb8-0769-46c4-b02f-e86d4c548f46 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.159449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.192767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.193393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.193668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.193977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.199836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbde66d-8039-4472-a894-076709cd3be3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.204309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:53.209125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:53.210087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:53.210909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 171/325] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:53.211568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.237379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.238053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.238335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.238618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.239720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.241163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.243075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 155/326] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:53.243917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.265786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.315205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa3a567-0fdd-46ef-973d-55f8281b7c17 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.353362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf35d3-e3e7-4e1f-b8ef-a965d3ad3b27 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.355150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.355821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.356103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.356380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.367135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:53.372353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:53.373919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:53.374947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 172/327] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:53.375781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.460690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.477908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.478187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.478470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.478879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.479549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:53.479852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.481223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.492964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 156/328] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:53.493819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.542010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ae349-2b36-4fd3-a098-aae3a8a4cfeb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.571432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.576450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.577069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.577353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.577631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] PATH_INFO: `/v3/groups` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.578288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.579759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.594237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0025937-83dc-44ec-a427-3aa71afef409 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.628945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.629549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.629851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.630128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.630780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:53.631084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.632530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.641441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 157/329] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:53.642544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.674887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 173/330] 10.222.0.52 () {60 vars in 1134 bytes} [Tue Aug 30 00:48:53 2022] POST /identity/v3/groups => generated 226 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:48:53.675826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.757396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a4bc9-618a-472e-9aac-3e232fb228d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.785867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.797062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.797706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.815379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.815379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] PATH_INFO: `/v3/users/91453f6e66e8402d919b2b44e638ff74` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.815379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] RBAC: Authorizing `identity:get_user(user_id=91453f6e66e8402d919b2b44e638ff74)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.815379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.815379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 158/331] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/users/91453f6e66e8402d919b2b44e638ff74 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:53.815791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cc4ca-be23-4aaa-8d4c-250e30c325fe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.815869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.820994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:53.858060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.859689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.859970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.860257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] PATH_INFO: `/v3/users/24cd5f9ff29e427f9bc91d175d60fd85` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.861913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] RBAC: Authorizing `identity:get_user(user_id=24cd5f9ff29e427f9bc91d175d60fd85)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.863400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:53.864989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 174/332] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/users/24cd5f9ff29e427f9bc91d175d60fd85 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:53.872377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:53.935623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f6611-06d5-45db-ae22-8a89c20dfb1e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:53.987582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:53.988515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:53.988824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:53.989086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:53.989857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] RBAC: Adding query filter params (role.id=f8e7032fac524859a7ef27220eb9a8fd, user.id=91453f6e66e8402d919b2b44e638ff74, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:53.990176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:53.992468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70644e52-d605-4272-bb01-34a13d328ff5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:54.008957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 159/333] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:53 2022] GET /identity/v3/role_assignments?user.id=91453f6e66e8402d919b2b44e638ff74&scope.system=all&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:48:54.010340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01f8e6-101a-494e-a720-9c824ccf805e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:54.056416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158e96f-8d7e-4768-8abf-a2efb429123d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:54.057150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158e96f-8d7e-4768-8abf-a2efb429123d None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:54.057443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158e96f-8d7e-4768-8abf-a2efb429123d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:54.057757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158e96f-8d7e-4768-8abf-a2efb429123d None admin] PATH_INFO: `/v3/system/users/24cd5f9ff29e427f9bc91d175d60fd85/roles/0ed49d44afa7491f8cf2b6f367dfcded` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:54.059454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158e96f-8d7e-4768-8abf-a2efb429123d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=24cd5f9ff29e427f9bc91d175d60fd85, role_id=0ed49d44afa7491f8cf2b6f367dfcded)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:54.061462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158e96f-8d7e-4768-8abf-a2efb429123d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:54.133212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 175/334] 10.222.0.52 () {58 vars in 1334 bytes} [Tue Aug 30 00:48:53 2022] PUT /identity/v3/system/users/24cd5f9ff29e427f9bc91d175d60fd85/roles/0ed49d44afa7491f8cf2b6f367dfcded => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:48:55.865739 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348315d-a6ba-4523-aaa3-9116a00886cd None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.866256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348315d-a6ba-4523-aaa3-9116a00886cd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.866358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348315d-a6ba-4523-aaa3-9116a00886cd None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.867928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 160/335] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:55.874290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a24b1-e7ff-4b49-be2e-75858c632a2b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:55.875211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a24b1-e7ff-4b49-be2e-75858c632a2b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:55.875692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9a24b1-e7ff-4b49-be2e-75858c632a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:55.898963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8b9a24b1-e7ff-4b49-be2e-75858c632a2b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:55.900487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8b9a24b1-e7ff-4b49-be2e-75858c632a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:55.914160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:55.992296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 176/336] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:56.344070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ee98a-b3bb-4e3d-b60f-0e5dce646a07 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.345777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0da46-91b8-4f44-90ad-37749b2afafc None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.346070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0da46-91b8-4f44-90ad-37749b2afafc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.346342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df0da46-91b8-4f44-90ad-37749b2afafc None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.348056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ee98a-b3bb-4e3d-b60f-0e5dce646a07 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.348337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ee98a-b3bb-4e3d-b60f-0e5dce646a07 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.349929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 177/337] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:56.350136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 161/338] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:56.352914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.353198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.353473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.377502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0a7e-c045-44d8-80ce-afaa94ed3729 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.377888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0a7e-c045-44d8-80ce-afaa94ed3729 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.378171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a0a7e-c045-44d8-80ce-afaa94ed3729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.379110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:56.380263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:56.414221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9c9a0a7e-c045-44d8-80ce-afaa94ed3729 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:56.416185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c9a0a7e-c045-44d8-80ce-afaa94ed3729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:56.572760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 178/339] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:48:56.573626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.587420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.602982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 162/340] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 00:48:56.661202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.679991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a9ad5-8e55-4755-8872-f154fd1b9722 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.680306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a9ad5-8e55-4755-8872-f154fd1b9722 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.680556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373a9ad5-8e55-4755-8872-f154fd1b9722 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.681930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 163/341] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:48:56.688681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3139186-802f-415b-afa0-307698e36199 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.688996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3139186-802f-415b-afa0-307698e36199 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.689242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3139186-802f-415b-afa0-307698e36199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.691298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b41f5e0-81c6-4d74-9904-9eafc971bff9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.722965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e3139186-802f-415b-afa0-307698e36199 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:56.724240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e3139186-802f-415b-afa0-307698e36199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:56.744728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.745488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.745800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.746081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.772916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:56.782141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:56.783105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:56.783902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 179/342] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:56.789690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:56.823272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 164/343] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:56.931730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a36c7-2c19-4bcd-b673-56a4d2ebba68 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:56.975961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:56.976809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:56.977256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:56.977673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:56.978524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:56.978972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:56.980489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:56.982236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:56.993882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 180/344] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:57.000115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3139186-802f-415b-afa0-307698e36199 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.041499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.124006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3139186-802f-415b-afa0-307698e36199 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.157768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec06ecfa-7058-4d33-8b82-a7db2d0abb89 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.189520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.210119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.211279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.211822 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.212264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.214738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.214738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.220866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.220866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.227926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:57.240288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.241216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:48:57.242035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:57.245896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 181/345] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.249372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.359381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.360924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.389101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.500981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee97284-9020-4ebd-a819-c4e78004cd47 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.546121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.546885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.547172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.547589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.560038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:57.565471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.566513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:57.567516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:57.569176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 182/346] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.574340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.606411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 165/347] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:57.612203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.692511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.704390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b8dd1a-9054-4099-8131-fa61e241c055 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.737572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:57.795219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.796039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.796329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.796613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.797296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:57.797608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:57.799052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:57.817327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 183/348] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:57.818235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.822620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc03b7-026a-442c-b9a4-bb394f00c0c3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:57.865414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:57.867131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:57.867536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:57.868243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:57.887521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:57.894887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:57.895814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:48:57.896694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 166/349] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:57.897349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:57.993600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645087fa-d9d2-4361-942f-89b8938ea1b6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.050199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.055989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.057233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.057988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.058414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.059243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:58.059761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.062465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.062600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f3c7c-67a9-4404-b896-318df7cb4714 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.073931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 184/350] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.075160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.113499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.115577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.116047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.117210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.118647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.120261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.121986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 167/351] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:57 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.127499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.136329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.230608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8793342-8bfb-4ffe-bb5b-2afc5a0b2e20 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.266553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ca64e5-26d8-480c-84bd-55a1880dfafa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.275032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.275726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.276013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.276290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.317557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.318520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.318974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.319387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.320351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:58.321587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.323208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298eff3d-e054-4146-86b4-51248bdb43b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.336781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 168/352] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.338562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c3de2-7309-4e80-a320-2c58a50b8663 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.339012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c3de2-7309-4e80-a320-2c58a50b8663 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.339768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c3de2-7309-4e80-a320-2c58a50b8663 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.341438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 169/353] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:58.343006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c3de2-7309-4e80-a320-2c58a50b8663 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.362054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.363540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.372175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.413380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.498329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 185/354] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:58.499618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.505755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c3de2-7309-4e80-a320-2c58a50b8663 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.544666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3220c-a9e7-426f-b5b1-bc4ed9d8a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.545288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3220c-a9e7-426f-b5b1-bc4ed9d8a2f6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.545570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3220c-a9e7-426f-b5b1-bc4ed9d8a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.545981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3220c-a9e7-426f-b5b1-bc4ed9d8a2f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.629753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3220c-a9e7-426f-b5b1-bc4ed9d8a2f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.632770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3220c-a9e7-426f-b5b1-bc4ed9d8a2f6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.633045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4b744-5cb3-4411-8202-8c8bf8873557 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.653340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.677093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.677657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.677989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.678264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.689846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.691961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.693550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 186/355] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.694258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.698464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.766943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 170/356] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:48:58.769607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.770985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.771183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.798059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:58.799601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:58.827799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aa5e91-f167-410d-894c-e7878a1e5d4d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:58.865872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533ebd19-6e4a-4296-b8e7-185b517a11cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:58.866517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ebd19-6e4a-4296-b8e7-185b517a11cd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.866804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ebd19-6e4a-4296-b8e7-185b517a11cd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.867084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533ebd19-6e4a-4296-b8e7-185b517a11cd None admin] PATH_INFO: `/v3/users/24cd5f9ff29e427f9bc91d175d60fd85` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.868248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ebd19-6e4a-4296-b8e7-185b517a11cd None admin] RBAC: Authorizing `identity:get_user(user_id=24cd5f9ff29e427f9bc91d175d60fd85)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:58.869693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533ebd19-6e4a-4296-b8e7-185b517a11cd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:58.876823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 187/357] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/users/24cd5f9ff29e427f9bc91d175d60fd85 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:58.878255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4195c43-24db-4387-af56-aafa660fe85d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:58.878544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4195c43-24db-4387-af56-aafa660fe85d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:58.878821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4195c43-24db-4387-af56-aafa660fe85d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:58.884885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 188/358] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:48:58.884885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4195c43-24db-4387-af56-aafa660fe85d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.885109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 171/359] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:58.885109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:58.941505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:58.970295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.011709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4195c43-24db-4387-af56-aafa660fe85d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.018241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197c410f-71de-42a1-8214-281aaf7f5a58 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.051744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.052377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.052662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.052942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.054120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.058866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.059584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 189/360] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.061777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.067021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.067828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.068061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.068436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.069770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=27f6325081de4a4bbd73323b17cd130a) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.069770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.071357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.078949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 172/361] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:58 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.079797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.207151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f552d95-5d3a-4feb-880d-77075444e8e8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.207325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c0fa50-9f49-434b-9cbc-9084518f055f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.222866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.259039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.259678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.259938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.260215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.260905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dda2a4-463a-45c9-b6a8-24517184add0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.261025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] RBAC: Adding query filter params (role.id=0ed49d44afa7491f8cf2b6f367dfcded, user.id=24cd5f9ff29e427f9bc91d175d60fd85, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.261307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.262072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda2a4-463a-45c9-b6a8-24517184add0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.262534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda2a4-463a-45c9-b6a8-24517184add0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.262779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.263197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dda2a4-463a-45c9-b6a8-24517184add0 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.265009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dda2a4-463a-45c9-b6a8-24517184add0 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.266595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dda2a4-463a-45c9-b6a8-24517184add0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.268449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 190/362] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.269483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.273839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.274391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.282659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 173/363] 10.222.0.52 () {56 vars in 1308 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/role_assignments?user.id=24cd5f9ff29e427f9bc91d175d60fd85&scope.system=all&role.id=0ed49d44afa7491f8cf2b6f367dfcded => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.283564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.294771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:48:59.296995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:48:59.299484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.384425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 191/364] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:48:59.385552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.445051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27de338-0dcb-42a1-96f6-6a6f5ac12783 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.535638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.536406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.536699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.536988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.538184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.539745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.541903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 174/365] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.542709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.552120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5917657-ad7e-4c71-980a-ce3b80c44f65 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.585554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.596856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.597497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.598542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.598869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.607570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.613466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:48:59.618986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:48:59.620105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:48:59.621128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:48:59.624894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 192/366] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:48:59.625821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.690224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996e1987-72c6-4b7a-a80e-1061cb4a6825 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.733908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.734804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.735264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.735689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.737809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.739429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.741228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 175/367] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.742185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.778235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614eaa5f-f31b-4e2e-8c35-1a9a79e979e3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.822386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.822997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.823273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.823546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.824781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.824781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.826743 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.833158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 193/368] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.834089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:48:59.907810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab73ab9-736d-4ea8-9b2c-fd10caf417b0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.910938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.913656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:48:59.953872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:48:59.954507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:48:59.954794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:48:59.955071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:48:59.955755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:48:59.956064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:48:59.957023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb08d29-b4da-4003-a21d-33a2a1493291 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:48:59.957513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:48:59.969259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 176/369] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:48:59.970227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.011839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.012347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.012442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.012685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.042794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:00.051146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:00.052452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:00.053711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:00.055550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 194/370] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:00.056827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.108203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6977411f-c072-4e58-ad06-ceef66a360fb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.158908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.159499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.159784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.160063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.160800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=27f6325081de4a4bbd73323b17cd130a) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.161115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.163819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.170033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 177/371] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:48:59 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=d0854046eb544eab8092d0a773a97724 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.170937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.209530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b3f78-f8fd-4fe7-a134-b4769a83870f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.217384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.240181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.255247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.255945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.256235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.256517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.333312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348a112b-8e70-484e-95c2-ac977538eb93 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.377641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.381995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.381995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.381995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.381995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.382301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:00.382301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.382301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.383353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.393904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 178/372] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.400159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.523422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 195/373] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:00.530085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.560967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.571356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.595984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fcb1a-a301-4b42-a7a7-36d6cdad0355 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.662820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b77121e-17c4-409d-a84f-c7ec1afe9d94 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:00.673874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.674672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.674964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.675287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.737843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:00.738784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:00.739235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:00.739704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:00.741093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.742963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.749818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 196/374] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:00.751401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.767750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:00.769723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:00.881011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.893705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 179/375] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:00.899349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:00.934826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:00.944235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f9ae6-6821-44cb-bdc9-17923e864131 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.003858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.005064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.005341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.005615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.009596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.011116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.021622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 197/376] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:01.040366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.121193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4977c4-b504-4a12-8386-206795d191e1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.184842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.185501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.186193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.186590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] PATH_INFO: `/v3/projects/dae85e39d46d422d97a780b67b09cea1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.188796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] RBAC: Authorizing `identity:get_project(project_id=dae85e39d46d422d97a780b67b09cea1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.192520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.194261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 180/377] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:00 2022] GET /identity/v3/projects/dae85e39d46d422d97a780b67b09cea1 => generated 344 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:01.198554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e4182-6492-4d45-8287-ed79507c9855 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.201479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.240799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.241698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.242198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.242655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.244030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:01.244541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.246774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.255066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 198/378] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=d0854046eb544eab8092d0a773a97724 => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:01.322840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.371021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51737337-abff-4feb-8766-6cfe68368cd0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.417615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10600581-31c5-4070-a79c-558bebf95cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.418310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10600581-31c5-4070-a79c-558bebf95cab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.418592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10600581-31c5-4070-a79c-558bebf95cab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.418912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10600581-31c5-4070-a79c-558bebf95cab None admin] PATH_INFO: `/v3/groups/f28e4df179134bc29e91e6dee760acaa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.426937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10600581-31c5-4070-a79c-558bebf95cab None admin] RBAC: Authorizing `identity:get_group(group_id=f28e4df179134bc29e91e6dee760acaa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.428409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10600581-31c5-4070-a79c-558bebf95cab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.430512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 181/379] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/groups/f28e4df179134bc29e91e6dee760acaa => generated 229 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:01.436782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:01.466164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:01.476364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ab464-af8c-408e-8c3c-0313d1bc1bd8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.476364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ab464-af8c-408e-8c3c-0313d1bc1bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.476732 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ab464-af8c-408e-8c3c-0313d1bc1bd8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.478432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 182/380] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:01.486943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.487232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.487514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.508818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:01.510046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:01.593418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 183/381] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:01.620670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05545a15-3390-4d3c-927a-e7c2d376a08f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:01.667761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:01.668386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:01.668676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:01.668955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:01.669760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] RBAC: Adding query filter params (group.id=f28e4df179134bc29e91e6dee760acaa, role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=dae85e39d46d422d97a780b67b09cea1) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:01.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:01.671658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:01.683194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 199/382] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/role_assignments?group.id=f28e4df179134bc29e91e6dee760acaa&scope.project.id=dae85e39d46d422d97a780b67b09cea1&role.id=9d7628878120443f8b9950c60d7a8671 => generated 260 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:01.933277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.003824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:02.069428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911b4e-36a1-496d-a48f-7d5aefdc7fd0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.111823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.112481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.112768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.113046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.128650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:02.133530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:02.134443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:02.135872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 184/383] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:02.140576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.268341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:02.287825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a291ac8-4cb6-4a56-bb2b-24b51b7634e9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.339062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.339687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.339978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.340509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.341235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:02.342493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:02.343903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b326a1-3331-45ca-8de1-9d43810d61ab None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:02.355611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 200/384] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:02.361361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.381429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68c34-36e3-4847-aaa5-b6137a080248 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.381782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68c34-36e3-4847-aaa5-b6137a080248 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.382067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e68c34-36e3-4847-aaa5-b6137a080248 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.385025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 201/385] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:02.393704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.394095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.394333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.424669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:02.428123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:02.569744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daaddd-2970-4167-8526-25b2c62a918e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.572277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 202/386] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:02.594971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:02.623568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.624221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.624563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.624853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.629943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:02.631476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:02.633065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 185/387] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:02.639590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.735613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:02.758862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a55779-b99f-4b0a-92c7-4532be7e1f45 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.798388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:02.799077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:02.799366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:02.799644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:02.800776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:02.802273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:02.804219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 203/388] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:02.811280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.875383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:02.981580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f93ca56-65f5-4161-85a1-c0e980620c9b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:02.989783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f77776e-70f4-44dc-a22a-68db24299990 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.028524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.029879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.030262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.030642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.042330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:03.047745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:03.048944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:03.049968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:03.051737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 204/389] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:03.057447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.061303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ec61a0-413c-4ed5-9fa6-c093b23e96e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.062280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec61a0-413c-4ed5-9fa6-c093b23e96e0 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.062772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec61a0-413c-4ed5-9fa6-c093b23e96e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.063252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ec61a0-413c-4ed5-9fa6-c093b23e96e0 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a/groups/e0a7cb2751b7493899fb6835ef83e827/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.065446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec61a0-413c-4ed5-9fa6-c093b23e96e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=27f6325081de4a4bbd73323b17cd130a, group_id=e0a7cb2751b7493899fb6835ef83e827, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.067233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ec61a0-413c-4ed5-9fa6-c093b23e96e0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.074296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.167027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 186/390] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 30 00:49:02 2022] PUT /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a/groups/e0a7cb2751b7493899fb6835ef83e827/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:03.173759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cedb2a-bd6d-4eea-b9ba-e4720661cb01 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.174535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cedb2a-bd6d-4eea-b9ba-e4720661cb01 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.174882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cedb2a-bd6d-4eea-b9ba-e4720661cb01 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.180901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 187/391] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:03.182827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.183192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.183620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.199587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9b36-c3de-40dd-b8db-918d4672c1fd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.233022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:03.235922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:03.270889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924f10-4148-4290-9e6d-5764575259da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.271513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924f10-4148-4290-9e6d-5764575259da None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.271796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924f10-4148-4290-9e6d-5764575259da None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.272071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07924f10-4148-4290-9e6d-5764575259da None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.272748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924f10-4148-4290-9e6d-5764575259da None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:03.273051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924f10-4148-4290-9e6d-5764575259da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.274477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07924f10-4148-4290-9e6d-5764575259da None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.294096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 205/392] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:03.299286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924f10-4148-4290-9e6d-5764575259da None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.326696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.393765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 188/393] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:03.419914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.457871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07924f10-4148-4290-9e6d-5764575259da None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.500238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b84e64-6198-4cd1-8c17-e26d6d3cfff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.500989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b84e64-6198-4cd1-8c17-e26d6d3cfff3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.501278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b84e64-6198-4cd1-8c17-e26d6d3cfff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.501553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b84e64-6198-4cd1-8c17-e26d6d3cfff3 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.502827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b84e64-6198-4cd1-8c17-e26d6d3cfff3 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.504280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b84e64-6198-4cd1-8c17-e26d6d3cfff3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.506256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 206/394] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:03.511476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.657862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04317c-c5f1-40f6-a77e-d3ae099de885 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.658254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04317c-c5f1-40f6-a77e-d3ae099de885 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.658430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc04317c-c5f1-40f6-a77e-d3ae099de885 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.660223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 207/395] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:03.665271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.665560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.665881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.683154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb8fba4-8da0-4e55-a25b-542f48a4bc80 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.691741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:03.693011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:03.770088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:03.771126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:03.771639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:03.772115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:03.773455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:03.783004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 208/396] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:49:03.783426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:03.785298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 189/397] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:03.795431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.795594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:03.872291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.926106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:03.980961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b66f4-7d00-4375-85c3-f1f6a6cf0a50 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:03.992089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a36fd-10d9-4a9a-a1d5-0cc7a87a28bb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.031726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a81e8-c0e7-4b6b-9748-9213cc3cda89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.032804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a81e8-c0e7-4b6b-9748-9213cc3cda89 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.033290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a81e8-c0e7-4b6b-9748-9213cc3cda89 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.033825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a81e8-c0e7-4b6b-9748-9213cc3cda89 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/groups/e0a7cb2751b7493899fb6835ef83e827/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.036312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a81e8-c0e7-4b6b-9748-9213cc3cda89 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2256eaae8634eaaabf64f763a14d6b7, group_id=e0a7cb2751b7493899fb6835ef83e827, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.038017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12a81e8-c0e7-4b6b-9748-9213cc3cda89 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.039828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da23d7f-e37b-48bd-99bc-fa96de9c036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.040472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da23d7f-e37b-48bd-99bc-fa96de9c036d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.040752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da23d7f-e37b-48bd-99bc-fa96de9c036d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.041064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da23d7f-e37b-48bd-99bc-fa96de9c036d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.053507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da23d7f-e37b-48bd-99bc-fa96de9c036d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0da23d7f-e37b-48bd-99bc-fa96de9c036d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:04.058456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.059388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:04.060334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:04.062462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 190/398] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.064121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afae55-ad2f-45e0-8046-a87bbd3e8b59 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.064627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afae55-ad2f-45e0-8046-a87bbd3e8b59 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.065130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5afae55-ad2f-45e0-8046-a87bbd3e8b59 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.067753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 191/399] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:04.069002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afae55-ad2f-45e0-8046-a87bbd3e8b59 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.120101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 209/400] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 30 00:49:03 2022] PUT /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/groups/e0a7cb2751b7493899fb6835ef83e827/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:04.121651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.122215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.122744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.152911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:04.155215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:04.224333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.229099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5afae55-ad2f-45e0-8046-a87bbd3e8b59 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.257334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 210/401] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:04.258173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.275638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.276704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.277213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.277698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.278671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.279308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.280989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.296607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 192/402] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.302361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.347406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.395164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cfd2e1-30dc-4158-865d-8df3f5850db3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.420559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad81a09-d4ad-453f-8221-e14ea4d8a584 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.460583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.461229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.461510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.461809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.463913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.464547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.464821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.465343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.468488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.468488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.470064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 193/403] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.478941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.494151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:04.499024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.500024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:04.500839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:04.504619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 211/404] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.508632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.548439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.630456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90d73ce-f48c-4a7d-a575-69432c57af44 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.646656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b80cb1c-e731-402a-b63b-4888877ffab8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.672061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.682607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.684389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.684869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.685450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.687746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.689701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.691636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 194/405] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:04.692588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.693065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.693409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.693874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.694234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.695100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:04.695479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.697030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.712018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 212/406] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:04.713014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:04.831005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:04.832996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca28769-3aad-4cee-932f-332d0c6b9e8c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.837009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802a315-5937-4a01-ab4c-bf71b5a0db67 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:04.874039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.875156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.875639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.876191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.880605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:04.881457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:04.882073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:04.882628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a/groups/e0a7cb2751b7493899fb6835ef83e827/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:04.885325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] RBAC: Authorizing `identity:create_grant(project_id=27f6325081de4a4bbd73323b17cd130a, group_id=e0a7cb2751b7493899fb6835ef83e827, role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:04.887401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:04.890005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:04.895483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:04.896568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:04.897541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:04.899430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 195/407] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:04.900088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.002849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 213/408] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 30 00:49:04 2022] PUT /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a/groups/e0a7cb2751b7493899fb6835ef83e827/roles/d0854046eb544eab8092d0a773a97724 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:05.004741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.024534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cfdd4d-e356-4012-975e-90b3106fb58b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.086541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.087132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.087416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.087699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.094319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.095860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.097563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 196/409] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:04 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.100014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.104643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.174081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.207554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0826a18-6a20-45a5-a946-53c458e4f8fc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.246320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.246942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.247225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.247553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.248233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:05.248551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.249961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.262765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 214/410] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.270838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.291890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a713b6ee-a40d-41a1-b172-aa5e4502e683 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.335817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.336750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.336980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.337264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.338584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.340594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.343628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 197/411] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.348817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.468731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.476246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2e8e0-3d11-4533-a0e7-636436bd522a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.515312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.520362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94091e5-14b0-4075-840d-2b789328da57 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.521417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0b65a-3d8d-47ad-abf8-53d1f4e529e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.522275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b65a-3d8d-47ad-abf8-53d1f4e529e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.522575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b65a-3d8d-47ad-abf8-53d1f4e529e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.522836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0b65a-3d8d-47ad-abf8-53d1f4e529e2 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/groups/e0a7cb2751b7493899fb6835ef83e827/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.525149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0b65a-3d8d-47ad-abf8-53d1f4e529e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=c2256eaae8634eaaabf64f763a14d6b7, group_id=e0a7cb2751b7493899fb6835ef83e827, role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.529674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0b65a-3d8d-47ad-abf8-53d1f4e529e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.562915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd3ed0-4471-4a80-99e5-17ef8ff00e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.563831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd3ed0-4471-4a80-99e5-17ef8ff00e25 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.564115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd3ed0-4471-4a80-99e5-17ef8ff00e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.564421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd3ed0-4471-4a80-99e5-17ef8ff00e25 None admin] PATH_INFO: `/v3/groups/f28e4df179134bc29e91e6dee760acaa` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.565571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd3ed0-4471-4a80-99e5-17ef8ff00e25 None admin] RBAC: Authorizing `identity:get_group(group_id=f28e4df179134bc29e91e6dee760acaa)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.567053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd3ed0-4471-4a80-99e5-17ef8ff00e25 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.568804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 215/412] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/groups/f28e4df179134bc29e91e6dee760acaa => generated 229 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.570025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841633e-d433-43fd-bcd6-38d157edbf53 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.570326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841633e-d433-43fd-bcd6-38d157edbf53 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.570610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841633e-d433-43fd-bcd6-38d157edbf53 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.576696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 216/413] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:05.577528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841633e-d433-43fd-bcd6-38d157edbf53 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.628859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 198/414] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 30 00:49:05 2022] PUT /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7/groups/e0a7cb2751b7493899fb6835ef83e827/roles/d0854046eb544eab8092d0a773a97724 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:05.630042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.630327 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.630604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.665131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:05.667995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:05.748692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841633e-d433-43fd-bcd6-38d157edbf53 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:05.799771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:05.800805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:05.801361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:05.801795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] PATH_INFO: `/v3/projects/dae85e39d46d422d97a780b67b09cea1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:05.805754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] RBAC: Authorizing `identity:get_project(project_id=dae85e39d46d422d97a780b67b09cea1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:05.805754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:05.806823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 217/415] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:05 2022] GET /identity/v3/projects/dae85e39d46d422d97a780b67b09cea1 => generated 344 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:05.813887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:05.836359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 199/416] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:05.865885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:05.965876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ab754-f8f4-4779-a289-978c974c3186 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.015641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.016607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.017072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.017527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] PATH_INFO: `/v3/projects/dae85e39d46d422d97a780b67b09cea1/groups/f28e4df179134bc29e91e6dee760acaa/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.023032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=dae85e39d46d422d97a780b67b09cea1, group_id=f28e4df179134bc29e91e6dee760acaa, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.024696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.106099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 218/417] 10.222.0.52 () {58 vars in 1442 bytes} [Tue Aug 30 00:49:05 2022] PUT /identity/v3/projects/dae85e39d46d422d97a780b67b09cea1/groups/f28e4df179134bc29e91e6dee760acaa/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:06.162540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.285721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.303642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40021a38-fe21-49bf-88c1-e9c6979a7c0e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.348959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb17e9-cdcf-488b-8824-acb6f2762490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.350005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb17e9-cdcf-488b-8824-acb6f2762490 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.350493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb17e9-cdcf-488b-8824-acb6f2762490 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.350968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb17e9-cdcf-488b-8824-acb6f2762490 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.364087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eb17e9-cdcf-488b-8824-acb6f2762490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-87eb17e9-cdcf-488b-8824-acb6f2762490 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:06.369394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:06.370364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:06.371170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:06.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 200/418] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:06.379484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.421796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3896f-dfee-48d7-ad3e-d6e58cb0a71f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.422438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3896f-dfee-48d7-ad3e-d6e58cb0a71f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.422883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3896f-dfee-48d7-ad3e-d6e58cb0a71f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.424985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 201/419] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:06.430397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.431003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.431509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.452956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:06.454655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:06.529020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 202/420] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:49:06.572139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951013ad-c6ed-4b4e-9d25-1ee6134536a1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.582583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.607968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.608572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.608851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.609123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.609809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] RBAC: Adding query filter params (name=member) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:06.610116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.611548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.620751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 219/421] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:06.626101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.708514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.736540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda05e70-a497-4772-bca1-c7eb29e717c7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:06.774041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:06.774674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:06.774958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:06.775236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:06.851365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:06.852749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:06.932512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:06.960675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 203/422] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:49:06.967453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:06.999788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.126669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa7a55-17bc-406c-8027-5cdf1ca3f85f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.188775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5072a24-2da6-4f48-a292-cf2eeddd099a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.191904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.192590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.192871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.193143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.195072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.196519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.199279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 220/423] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:07.205356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.251139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.262324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.262959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.263796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.264076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.285017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:07.290789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:07.291922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 00:49:07.292798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:07.296207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 204/424] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:07.300939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.343251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.412121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ef30d-fea1-4f9f-9145-edeb99b0976c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.433063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c5ef9c-3441-40b6-a4fa-f0c6a10787b1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.499796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.500437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.500723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.501002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.501690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:07.502027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.503449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.515694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.516346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.516630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.516909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.518407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.519942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.523286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 221/425] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:07.525616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 205/426] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:07.527648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.533197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:07.665659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.668189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:07.675518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210924e-9f30-4bd6-86f9-6d8b229031af None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.676363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4a9ea-eaa7-420f-8ef5-18c7352760e2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:07.721037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c4772-aeef-4150-8c05-085857d0ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.721658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c4772-aeef-4150-8c05-085857d0ade4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.721965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:07.722091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c4772-aeef-4150-8c05-085857d0ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.722246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c4772-aeef-4150-8c05-085857d0ade4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.723112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.723581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.724029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.725159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=27f6325081de4a4bbd73323b17cd130a) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:07.725648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.728128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a92d12-45c0-4bbe-acb1-aa784495e7a8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.736615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 222/427] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:07.738176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f455-fb23-4177-a894-a9deea2f8b35 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.738649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f455-fb23-4177-a894-a9deea2f8b35 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.739100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f455-fb23-4177-a894-a9deea2f8b35 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.745604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 223/428] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:07.748186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:07.749380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:07.750036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:07.787763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:07.789757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:07.880835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c4772-aeef-4150-8c05-085857d0ade4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:07.882375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c4772-aeef-4150-8c05-085857d0ade4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:07.924373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 224/429] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:08.012652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.020666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 206/430] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:08.026292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.034385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb69ff8-afd0-4e62-bac9-027d7cbc62e4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.034670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb69ff8-afd0-4e62-bac9-027d7cbc62e4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.034946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb69ff8-afd0-4e62-bac9-027d7cbc62e4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.037554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 207/431] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:08.041678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.042010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.042292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.061785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:08.062983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:08.145418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 208/432] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:49:08.158302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.163318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe1414-96f7-4278-b6cf-c746008e8778 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.199085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.199833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.200314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.201324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.203587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:08.206405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:08.211286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 225/433] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:08.216046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.238723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.343553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84ee45-7de5-4aea-9b79-fcbdc6807ee3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.375386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.379033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf6d2c-ff16-4bf1-9014-480bfc27717a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.389515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.390949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.391238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.391520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.392719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:08.394260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:08.398141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 209/434] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:08.406847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.417879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.419016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.419512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.419712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.434093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:08.441590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:08.442509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:08.443896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 226/435] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:08.452203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.480458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.548960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735ae611-c858-4b19-a09e-97857e993826 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.588639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2f162-18ea-4e7f-a167-7bcc0953e46a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.592465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.593168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.593454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.593748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.594501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=f8e7032fac524859a7ef27220eb9a8fd, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:08.594820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:08.596266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:08.602612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 210/436] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=f8e7032fac524859a7ef27220eb9a8fd => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:08.603436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.633819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.634454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.634737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.635020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.635713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:08.636041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:08.637457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81cf69e-53f9-45a6-8b01-59869821303c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:08.647274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 227/437] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:08.648567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b0e0-3b7a-439c-85ba-b862f40e6bd2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.648858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b0e0-3b7a-439c-85ba-b862f40e6bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.649136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e55b0e0-3b7a-439c-85ba-b862f40e6bd2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.655312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 228/438] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:08.655312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b0e0-3b7a-439c-85ba-b862f40e6bd2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.679609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.729361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c9e70-5c65-4bb9-8d24-0de5a7b34375 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.750468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.768730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ccc942-f19d-41b4-afb9-4f2263d3d3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.769333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccc942-f19d-41b4-afb9-4f2263d3d3db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.770476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccc942-f19d-41b4-afb9-4f2263d3d3db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.770762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ccc942-f19d-41b4-afb9-4f2263d3d3db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.777336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e55b0e0-3b7a-439c-85ba-b862f40e6bd2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:08.786200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ccc942-f19d-41b4-afb9-4f2263d3d3db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-65ccc942-f19d-41b4-afb9-4f2263d3d3db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:08.791561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:08.792436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:08.793214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 211/439] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:08.793943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.794506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.794506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.813290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:08.814557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:08.824260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:08.825100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:08.826147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:08.826601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:08.883758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 212/440] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:08.884619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:08.895771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:08.897229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:08.949334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:08.994462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.003890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5b5ad-5af6-4b9f-bd5c-9f9fee46a174 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.021679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 229/441] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:09.026981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.047669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.048296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.048581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.049089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.049532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.049874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.051259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.060917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 213/442] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.066416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.145526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec81f5c-73f9-4f42-9496-74a27c1995f0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.181018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.181267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb894c0-1c14-4c15-a6c5-e1c80b4c2c48 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.181652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.181966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.182242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] PATH_INFO: `/v3/projects/27f6325081de4a4bbd73323b17cd130a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.183382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] RBAC: Authorizing `identity:get_project(project_id=27f6325081de4a4bbd73323b17cd130a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.185347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.187208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 230/443] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/projects/27f6325081de4a4bbd73323b17cd130a => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:09.187988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.193465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.218007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.218618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.218901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.219177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.243160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.297387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.298910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.301918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75225023-9a0e-441e-8720-99f008412b1e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.336290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.336920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.337207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.337480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.348812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:09.353616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:09.355133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 231/444] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:09.355932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.403680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 214/445] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:49:09.405123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.416849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.459549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99721fc1-3ecf-4edc-9734-6e39aeb0cd10 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.467340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.495196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.495830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.496110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.496386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.497920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.499282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.501275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 232/446] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.501723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.515795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f2206-895a-4d00-95cf-46ff9465e3bc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.551580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.552092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.552377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.552653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.553315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.553627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.555075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.563752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 215/447] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.564332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.609543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5913facc-f08f-49b1-aa3a-ae5131d10743 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.635681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.645508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.656174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.656174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.656174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] PATH_INFO: `/v3/projects/c2256eaae8634eaaabf64f763a14d6b7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.656174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] RBAC: Authorizing `identity:get_project(project_id=c2256eaae8634eaaabf64f763a14d6b7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.656174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.656174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 233/448] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/projects/c2256eaae8634eaaabf64f763a14d6b7 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.656641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.666943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fccbe8-aee5-4e9a-9b6d-b59abf62cc0f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.696227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.702789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.703408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.703689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.703962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.704683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=27f6325081de4a4bbd73323b17cd130a) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:09.704998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.706468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.712331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 216/449] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=27f6325081de4a4bbd73323b17cd130a&role.id=d0854046eb544eab8092d0a773a97724 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.713124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.759787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d310a637-e864-4aa0-a037-863dea0f2980 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.811635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.812590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.812874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.813154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.852848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dccf18-e9b7-44d5-a763-d9ca52c93760 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:09.896247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.898726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.899967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:09.900618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:09.901089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:09.901376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] PATH_INFO: `/v3/groups/e0a7cb2751b7493899fb6835ef83e827` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:09.904070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] RBAC: Authorizing `identity:get_group(group_id=e0a7cb2751b7493899fb6835ef83e827)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:09.905752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:09.907389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 217/450] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/groups/e0a7cb2751b7493899fb6835ef83e827 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:09.914874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:09.921407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:09.968191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.053693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c82cc0-32ad-4ff9-9705-bbf9a8dad825 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.063111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 234/451] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:10.069891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.107838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.108822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.109245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.109674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.110644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] RBAC: Adding query filter params (group.id=e0a7cb2751b7493899fb6835ef83e827, role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=c2256eaae8634eaaabf64f763a14d6b7) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:10.111096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.113032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.123353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 218/452] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:09 2022] GET /identity/v3/role_assignments?group.id=e0a7cb2751b7493899fb6835ef83e827&scope.project.id=c2256eaae8634eaaabf64f763a14d6b7&role.id=d0854046eb544eab8092d0a773a97724 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:10.219636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69435f6a-9944-47b3-806e-fd13970e10ce None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.263604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.264340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.264629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.264903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] PATH_INFO: `/v3/projects/dae85e39d46d422d97a780b67b09cea1` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.266248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] RBAC: Authorizing `identity:get_project(project_id=dae85e39d46d422d97a780b67b09cea1)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.267698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.269885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 235/453] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/projects/dae85e39d46d422d97a780b67b09cea1 => generated 344 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:10.275838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.378804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.424677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1698a-a4bd-4700-ac77-3b0cc4f03b54 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.467096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6c1db-89b8-4e4e-81bb-c294da2a2cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.467921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c1db-89b8-4e4e-81bb-c294da2a2cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.468230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c1db-89b8-4e4e-81bb-c294da2a2cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.468502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6c1db-89b8-4e4e-81bb-c294da2a2cc8 None admin] PATH_INFO: `/v3/groups/f28e4df179134bc29e91e6dee760acaa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.469914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6c1db-89b8-4e4e-81bb-c294da2a2cc8 None admin] RBAC: Authorizing `identity:get_group(group_id=f28e4df179134bc29e91e6dee760acaa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.472346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6c1db-89b8-4e4e-81bb-c294da2a2cc8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.473146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 219/454] 10.222.0.52 () {56 vars in 1180 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/groups/f28e4df179134bc29e91e6dee760acaa => generated 229 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:10.479414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:10.497934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:10.603181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3536d16a-8540-4804-a1ed-6edb8dbf2f30 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:10.642650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:10.643578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:10.644087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:10.644376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:10.645197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] RBAC: Adding query filter params (group.id=f28e4df179134bc29e91e6dee760acaa, role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=dae85e39d46d422d97a780b67b09cea1) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:10.645782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:10.647809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693c9d-2ae6-40a2-9cd2-c4370da9c42d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:10.654405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 236/455] 10.222.0.52 () {56 vars in 1376 bytes} [Tue Aug 30 00:49:10 2022] GET /identity/v3/role_assignments?group.id=f28e4df179134bc29e91e6dee760acaa&scope.project.id=dae85e39d46d422d97a780b67b09cea1&role.id=9d7628878120443f8b9950c60d7a8671 => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:13.209262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b8899-609f-4f7d-b055-aef300961ff4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.209815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b8899-609f-4f7d-b055-aef300961ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.209941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b8899-609f-4f7d-b055-aef300961ff4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.211338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 220/456] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:13.223241 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c857b07-2288-44eb-84b6-f3920132d0a0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.223721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c857b07-2288-44eb-84b6-f3920132d0a0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.223795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c857b07-2288-44eb-84b6-f3920132d0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.247445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3c857b07-2288-44eb-84b6-f3920132d0a0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:13.248899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c857b07-2288-44eb-84b6-f3920132d0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:13.339411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 237/457] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:13.381464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895fcd3-8fd1-45b0-8f32-25e9e6bb20c9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.381803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895fcd3-8fd1-45b0-8f32-25e9e6bb20c9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.382099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895fcd3-8fd1-45b0-8f32-25e9e6bb20c9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.383756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 221/458] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:13.389577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e0cb4-25d5-4bbe-855d-6d0eebc141c8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.389930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e0cb4-25d5-4bbe-855d-6d0eebc141c8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.390167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e0cb4-25d5-4bbe-855d-6d0eebc141c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.402308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.422979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c05e0cb4-25d5-4bbe-855d-6d0eebc141c8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:13.424178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c05e0cb4-25d5-4bbe-855d-6d0eebc141c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:13.519584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 238/459] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:13.559312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a6df9-f633-4e5d-b634-d1a0e2e30b80 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.560199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a6df9-f633-4e5d-b634-d1a0e2e30b80 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.560405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a6df9-f633-4e5d-b634-d1a0e2e30b80 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.562087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 222/460] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:13.567160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.567452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.567734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.594896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:13.596105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:13.681069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a22e4e-3aa9-43dc-8aa1-b2844c8ccb63 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.681386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a22e4e-3aa9-43dc-8aa1-b2844c8ccb63 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.681634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a22e4e-3aa9-43dc-8aa1-b2844c8ccb63 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.683290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 223/461] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:13.689055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:13.689364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:13.689627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:13.700025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 239/462] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:13.722034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:13.723262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:13.822285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 224/463] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:49:13.823787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.839398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:13.941332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:13.959760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.014121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644aa48a-7374-44b6-b788-bf0a645b2fca None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.023836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd82663-d92c-44e8-af5f-70dfc038519d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.052081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.052726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.053065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.053348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.062940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.063594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.063873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.064150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.064850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.066493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.076138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.077527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:14.082506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:14.083411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:14.084267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:14.084267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.084267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:14.084267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 240/464] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:14.089400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:14.094027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:14.096010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:14.098095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:14.098095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.098095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:14.101797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 225/465] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:14.101797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.197889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5537a5c-228f-4255-8b43-2ddd7f87ed06 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.215304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.219578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f2b2e-ea8e-4d9f-8896-666e6220f6f2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.234959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.235600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.235886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.236165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.239525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.248646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.250179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.259567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.260495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.260926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.261350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.262267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.264636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.264840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:14.266441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:14.267514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:14.268384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:14.268384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.268384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:14.268384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 241/466] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:14.268711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.287366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 226/467] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.288652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.387518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c1535-7af1-4b7d-b6c8-f2e3678d9b76 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.403237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd662910-8582-46fb-8d11-29173b73a08f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.424977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.425581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.425891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.426169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.426807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.427134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.428517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.443112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 242/468] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.444239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.449871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.451297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.451614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.451892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.469131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.472040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.476560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:14.494373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:14.499845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:14.500716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:14.500716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:14.500716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:14.500716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 227/469] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:14.500716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.531223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.564165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830fa11a-a1f5-46db-bda1-203ec0bfd085 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.602981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.603591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.603871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.604147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.604795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.605097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.606576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.610441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a241edb-8c84-4781-bec1-ddcd67ed07f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.618239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 243/470] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.619028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.644718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.645247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.645526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.645830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.646473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.647912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.708681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.721462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c853899-c070-4b61-aab1-9db85e22dbca None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.736620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 228/471] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:14 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:14.737462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.757159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2429c1-57b8-4757-a9d8-8bb2af9febda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.757815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2429c1-57b8-4757-a9d8-8bb2af9febda None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.758101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2429c1-57b8-4757-a9d8-8bb2af9febda None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.758377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2429c1-57b8-4757-a9d8-8bb2af9febda None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.759110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2429c1-57b8-4757-a9d8-8bb2af9febda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.760619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2429c1-57b8-4757-a9d8-8bb2af9febda None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.842636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:14.859344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094af172-f282-4d67-a403-3fde1965e0dc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:14.888728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 244/472] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:14 2022] POST /identity/v3/users => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:14.896854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:14.897410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.897698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.898073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.898771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:14.899027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0321c9-d6c6-474f-a7da-fc084708288b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.899121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:14.899525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0321c9-d6c6-474f-a7da-fc084708288b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.899931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0321c9-d6c6-474f-a7da-fc084708288b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.900523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d80da3-e3a7-42e5-9229-823a7bf5de53 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:14.901898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 245/473] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:14 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:14.902529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0321c9-d6c6-474f-a7da-fc084708288b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:14.920207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 229/474] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:14.921511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:14.921918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:14.922258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:14.946074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:14.947353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:15.042504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 230/475] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:15.043488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.061082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0321c9-d6c6-474f-a7da-fc084708288b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.108462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.109319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.110240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.110660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.111734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.113910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.137125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.166777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fced0-84c2-4f8a-add5-98725fef02ee None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.203979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.205995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.206305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.206636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.207599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.208551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.209539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.214282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 246/476] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:14 2022] POST /identity/v3/users => generated 248 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:15.327377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 231/477] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:15 2022] POST /identity/v3/users => generated 249 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:15.343351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.497251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e213ad3-274d-44eb-a111-2c70cb2f324b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.546741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.547361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.547641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.547911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.548576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:15.549991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:15.597756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.sql.core [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:49:15.597756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 00:49:15.597756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [parameters: {'id': '5a4faa7e199d48548b1358cb9a62a5ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 00:49:15.597756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81621) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 30 00:49:15.613617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:49:15.614601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application yield resource Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise self.value Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application session.commit() Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:49:15.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:49:15.616440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:49:15.618343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:49:15.621236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [parameters: {'id': '5a4faa7e199d48548b1358cb9a62a5ed', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:15.623518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 00:49:15.625488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:15.634584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 247/478] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 30 00:49:15 2022] POST /identity/v3/roles => generated 144 bytes in 285 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:15.634711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.753219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836e18f1-c895-4f9f-9567-da182acabd2d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:15.794524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.800599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d4d90b-20ab-487e-bcdc-9c623af86969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:15.801258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4d90b-20ab-487e-bcdc-9c623af86969 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:15.801604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4d90b-20ab-487e-bcdc-9c623af86969 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:15.801916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d4d90b-20ab-487e-bcdc-9c623af86969 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:15.817604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d4d90b-20ab-487e-bcdc-9c623af86969 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-50d4d90b-20ab-487e-bcdc-9c623af86969 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:15.822580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:15.823477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:15.824473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 232/479] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:15.830085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:15.911054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:15.961127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4273b58-abb6-4ed7-9954-1b713586b6db None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:16.000951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:16.001646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:16.001985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:16.002264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:16.002960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:16.003325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:16.004793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc136b72-75c7-4aeb-8df1-9bee139576fc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:16.015011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 248/480] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:17.155806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d5f7a-5749-42b0-92e7-c95e202582de None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.156348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d5f7a-5749-42b0-92e7-c95e202582de None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.156427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d5f7a-5749-42b0-92e7-c95e202582de None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.157952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 233/481] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:17.163164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72cd89-e7af-487e-a0d3-fe9bf1bc3be3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.163648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72cd89-e7af-487e-a0d3-fe9bf1bc3be3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.163800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db72cd89-e7af-487e-a0d3-fe9bf1bc3be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.192094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-db72cd89-e7af-487e-a0d3-fe9bf1bc3be3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:17.193308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-db72cd89-e7af-487e-a0d3-fe9bf1bc3be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:17.284547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 249/482] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:17.513945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1648c4-d498-451a-bc4a-7b527a5f9ea9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.514316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1648c4-d498-451a-bc4a-7b527a5f9ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.514474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1648c4-d498-451a-bc4a-7b527a5f9ea9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.516100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 234/483] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:17.521245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aabb7-d91f-45c2-a66f-272ac28ac929 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.521532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aabb7-d91f-45c2-a66f-272ac28ac929 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.521826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51aabb7-d91f-45c2-a66f-272ac28ac929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.550714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b51aabb7-d91f-45c2-a66f-272ac28ac929 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:17.552197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b51aabb7-d91f-45c2-a66f-272ac28ac929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:17.624827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:17.652794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 250/484] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:17.703374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1648c4-d498-451a-bc4a-7b527a5f9ea9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.854571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1648c4-d498-451a-bc4a-7b527a5f9ea9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:17.865283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23427041-08c0-4a77-9b8f-8edb62a5bc3f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.865654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23427041-08c0-4a77-9b8f-8edb62a5bc3f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.865858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23427041-08c0-4a77-9b8f-8edb62a5bc3f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.867497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 251/485] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:17.873826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6bda-b110-45de-b837-a4a09393c3d9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.874148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6bda-b110-45de-b837-a4a09393c3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.874386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6bda-b110-45de-b837-a4a09393c3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.905701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-06ea6bda-b110-45de-b837-a4a09393c3d9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:17.906958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-06ea6bda-b110-45de-b837-a4a09393c3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:17.914170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefe6a3-e93c-495e-88a9-dcb6c10a3004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:17.914818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe6a3-e93c-495e-88a9-dcb6c10a3004 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.915105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe6a3-e93c-495e-88a9-dcb6c10a3004 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.915387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefe6a3-e93c-495e-88a9-dcb6c10a3004 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.930952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefe6a3-e93c-495e-88a9-dcb6c10a3004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8eefe6a3-e93c-495e-88a9-dcb6c10a3004 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:17.937946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:17.938907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:17.939715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:17.940492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 235/486] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:17.944657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05487e-d5db-4efd-b26e-f0eb0b949008 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:17.944657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05487e-d5db-4efd-b26e-f0eb0b949008 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:17.944657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05487e-d5db-4efd-b26e-f0eb0b949008 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:17.944657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 236/487] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:17.950912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05487e-d5db-4efd-b26e-f0eb0b949008 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:17.993593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 252/488] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:49:18.002001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.002001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.002001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.015230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.026668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:18.027892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:18.074561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05487e-d5db-4efd-b26e-f0eb0b949008 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.122702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 253/489] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:18.122702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.125674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.126976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.126976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.126976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.127423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:18.127733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:18.129115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:18.147019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 237/490] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:18.156589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.216898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.293009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc961f1-0761-4b2c-9c70-4deab7a10a2e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.308615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.330509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a35fcc4-95b5-40cc-b4d2-9e5364e35d14 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.330820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.331512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.331802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.332081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.345036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:18.346553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:18.362836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:18.371045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:18.372205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:18.372205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:18.372205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:18.372205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 238/491] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:18.374835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.402278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.403922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.404583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.405235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.421795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:18.427544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:18.428451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:18.429233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:18.431268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 254/492] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:18.431943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.524792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3eb565-47ba-4957-a188-7a84784bcd3c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.553027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4485a3ac-332f-4a19-acc5-338631822205 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.560378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.560993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.561276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.561555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.562122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.562249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:18.562533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:18.563939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:18.576254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 239/493] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:18.578341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.593606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.594275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.594556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.594826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.606958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:18.611718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:18.612739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 255/494] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:18.618592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.619943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.710630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb6518-d411-482b-ba45-52c18578e7ad None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.753264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b9a98a-c243-40e8-a4dd-9baffcb4fe73 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.758248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.758856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.759141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.759420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.760086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:18.760392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:18.761790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aac5c4a-9ec2-4fea-9bb7-3d5ef9f51696 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:18.770928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 240/495] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:18.772156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5b4bd-0add-43a9-8c94-9308f59a1fa1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.772442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5b4bd-0add-43a9-8c94-9308f59a1fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.772722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5b4bd-0add-43a9-8c94-9308f59a1fa1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.773936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 241/496] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:18.774629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5b4bd-0add-43a9-8c94-9308f59a1fa1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.797504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.798180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.798458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.798724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.810654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:18.816349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:18.817358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:18.818213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 256/497] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:18.818877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.848945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.874002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:18.887202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5b4bd-0add-43a9-8c94-9308f59a1fa1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.924248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.924904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.925186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.925463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.928654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664be6d4-cd47-44d4-897b-6721c96dae1d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:18.933688 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:18.936468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:18.949944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:18.952203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 242/498] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:18.953221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:18.971918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:18.972442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.972718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.972990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:18.973662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:18.973993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:18.975373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53180e-b8a1-430c-ba90-7adbb20b776b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:18.986473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 257/499] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:18.987710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:18.987994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:18.988271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.006663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:19.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:19.063265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66628baf-d270-46d6-9b9f-3f59ef8013c7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.083734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 258/500] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:19.084747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.103942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.111221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.111980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.112294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.112636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.126111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.127587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.136683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:19.144279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:19.145475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:19.146473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:19.146473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.146473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:19.146473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 243/501] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:19.151146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.198113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c1a22-84b9-4966-a7a0-b2d9103ea81a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.234945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.236282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.236896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.237233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.237955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:19.238269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.239805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.254231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 259/502] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:19.254231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.268081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68596bad-7588-4f38-be93-f27bddb5fce6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.305721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.306531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.306850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.307078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.307866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:19.308107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.309648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.347533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.350104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 244/503] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:19.354800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.368757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f8a92-00ac-4209-a31f-00150d9a32b2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.407344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.407967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.408244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.408513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.415171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.419944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.421424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:19.435650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:19.436592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:19.437441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:19.437441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.437441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:19.437441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 260/504] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:19.438181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.463406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17826458-5865-4b0d-a68e-cf31747b9f84 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.498698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.499324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.499612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.499892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.500543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:19.500851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.502267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.514077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 245/505] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:19.514903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.541571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1b398-e4ff-4bd1-9a0e-225352c94008 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.571066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.576078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.576700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.576979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.577256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.588397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.590145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:19.604634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:19.605583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:19.607143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:19.607143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.607143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:19.608099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 261/506] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:19.609077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.619684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f4883-1f1b-4b00-a55b-6415d51b8bdc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.637606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.655054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.655734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.655951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.656226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.712944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c74123-3c90-4f53-8631-0ede098db0bd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.725387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.726900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.749528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.750115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.750398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.750674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.764669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.766180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:19.784439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:19.785642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:19.786746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:19.786746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:19.786746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:19.786746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 262/507] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:19.786746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.822834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.838143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 246/508] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:19.839047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.861100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:19.890708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53296017-9944-4c4c-9555-6b389b50a7da None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.924544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.925154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.925441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.925719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.926389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:19.926696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.928083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:19.940085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 263/509] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:19.942842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99df6-a3ae-4755-9b50-aa40fd15ebd3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:19.943864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:19.977994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:19.978625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:19.978909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:19.979187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:19.986668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:19.988132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:20.000056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.001047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:20.001979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:20.005691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 247/510] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.006566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.045675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.051781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4264fb-ac34-4c8f-9f27-c802ff70973d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.093526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.094648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.095271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.095553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.095902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.097146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.097657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.099384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.112032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 264/511] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.112868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.116404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b590fd85-9097-4091-acc2-7077ef442f10 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.158643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.159264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.159549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.159828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.160476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.160789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.162203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.174999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 248/512] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.175820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.217503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824a4070-5e3a-4014-8119-9f6246411ed5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.254088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.254794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.254928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.255200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.267435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.268932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:20.284986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.286006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:20.286850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:20.286850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:20.286850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:20.286850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 265/513] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.287080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.298038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.300299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40efc8a-2759-4472-a9c1-65d8d2f28417 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.335681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.336301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.336578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.336855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.344437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.345907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.348236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:20.359419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.366341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 249/514] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.367255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.403058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2ba52-62c0-4198-974e-65e24b73ec25 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.450769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.452145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.452578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.452902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.453751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.454072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.455527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.474354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bb0b2-b52c-45aa-bc94-297fbd5431bf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.493212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 266/515] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.494084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.513284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.513852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.514132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.514405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.522044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.523500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:20.534608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:20.535554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 250/516] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:20.536423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.564904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.596877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.598729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188adc19-458d-4ef4-a008-99562c7392e1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.634356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.634971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.635359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.635523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.636171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.637696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.641050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c25178f-1f97-4bb3-9170-f87f0afe2967 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.675542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.676149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.676433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.676711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.677352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.677660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.679411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.691721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 251/517] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.692552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.760852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 267/518] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:20 2022] POST /identity/v3/users => generated 247 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:20.761826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.801746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ec58ef-0ab4-4cc4-98bf-19a7f175b912 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.824827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.845866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.847449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.847939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.848404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.849434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:20.850125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:20.852786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:20.881717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 252/519] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:20.882975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:20.898309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb326dd5-745c-4b76-9d5e-f466751f4b76 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:20.926600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:20.940143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:20.941251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:20.941548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:20.942515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:20.996200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968f28e-114d-44cc-873c-649cad8e32a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.024809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.026647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.032085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.032695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.032976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.033245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.034009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:21.034315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.035744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.055373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 253/520] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:21.056719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.097069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.141210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 268/521] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:21.142211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.174937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca79a01-029a-4679-bf7c-b4a3ede004d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.175160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.210971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.211584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.211863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.212138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.220243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.221691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:21.235147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:21.236268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:21.237361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:21.240890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 254/522] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:21.241707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.263697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b64162-9fed-4f1b-8aa7-59f0759e89e3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.299864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.300631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.301016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.301391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.361520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.363032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7258dde-9679-4ef4-b259-89550ae0d1e9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.383461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.390310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.401926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.402770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.403290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.404405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.410914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.488032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.489678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.501021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 269/523] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:21.502550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.607608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.614809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316a0149-bf00-47fd-962a-0f5fc082fca5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.620230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 255/524] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:21.620230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.643757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.656585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.657224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.657502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.657801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.668172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.669932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:21.686329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:21.687395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:21.688235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:21.688235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.688235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:21.688235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 270/525] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:21.688235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.743051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f418f0-64ba-4cf2-a1bc-0e32eb217647 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.783265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.783901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.784194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.784480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.785192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:21.785509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.786985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.804460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0502f9cd-34b4-4317-a9ae-d2c1b4754f65 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.806247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 256/526] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:21.806922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.860130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.861116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.861471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.861766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.871422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.875794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.877468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:21.895954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:21.896983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:21.897885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:21.897885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.897885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:21.897885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 271/527] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:21.897885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:21.926881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8621d3c-af81-4aae-8ab7-5d24ad511a7a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:21.933785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:21.964172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:21.965067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:21.965398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:21.966368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:21.978627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:21.980099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:21.995524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:21.996658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:21.998015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:21.998015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:21.998015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:21.998015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 257/528] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:21.998015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.016290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6718d7-a1a2-41ac-8cbb-93a8d6d44d9b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.054957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.056166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.056445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.056617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.057292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.057602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.059010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.076272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 272/529] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.076272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.119582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b641066-ca7f-4d1f-9e0b-6c78280c415b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.127454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.157823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.158509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.158846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.159121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.160199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=d0f8fc283cce4eff8126074e2a2c39fa) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.160818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.162612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.164856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.168732 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 258/530] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:21 2022] GET /identity/v3/role_assignments?user.id=d0f8fc283cce4eff8126074e2a2c39fa&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.169626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.196285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22798e6d-5818-48b5-9dd7-a6abb9b7e3a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.245037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.246087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.246600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.247221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.248480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.249291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.251247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.269200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 273/531] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.271378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.316945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f46bf8-d978-4d52-8779-c76e7d2d526b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.375945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.376580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.376822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.377094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.377903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.378077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.379450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.399121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 259/532] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.400237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.405996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927c84b-1850-4507-866e-bc96a4b843be None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.444471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.445114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.445387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.445658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.445915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.454355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.462368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:22.471964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:22.472877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:22.473798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 274/533] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:22.474275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.484507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.540602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cb1407-819a-40ec-a341-9abf680ea2ad None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.586674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.587319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.587598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.588192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.594278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4581d87-fe0a-4b98-96b2-21bff4853ed0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.596694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.599015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:22.613510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:22.614429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:22.615368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 260/534] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:22.615852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.641235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.642207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.642705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.643132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.651697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.653567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:22.670203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:22.671124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:22.672132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 275/535] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:22.672701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.741653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.751672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0edbca-ff56-44c6-8e88-d109c264a2bb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.756115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:22.789521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.790234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.790515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.790788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.791482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.791788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.793303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.804065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08bffb-b469-472d-8ccc-a4765a095730 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.806121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 261/536] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.807006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.849104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.850160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.850448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.850719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.851762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.852076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.853953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4466b5-0cab-4171-8e4d-394fe139367a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.877462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 276/537] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.878830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a748-bd8c-4018-859b-b8a094bb232d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.879432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a748-bd8c-4018-859b-b8a094bb232d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.879432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370a748-bd8c-4018-859b-b8a094bb232d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.880872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 277/538] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:22.881606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a748-bd8c-4018-859b-b8a094bb232d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.922480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cb225-10e8-49db-a57d-809bcfbc68d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:22.958576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:22.959336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:22.959721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:22.960003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:22.960703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:22.961007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:22.962474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:22.975321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 262/539] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:22.975949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:22.990390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.005229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370a748-bd8c-4018-859b-b8a094bb232d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.032704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.042339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.042953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.043239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.043519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.045054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=e1a623f1fc434449a76106e74756e35c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:23.045384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:23.046877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd78e42-4543-41d4-9e43-930a9828e34d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:23.052290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 278/540] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/role_assignments?user.id=e1a623f1fc434449a76106e74756e35c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:23.053682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.054022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.054303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.076590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:23.077934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:23.113894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce532b4-2eee-4306-8498-ca593d5a48ec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.165412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.166202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 279/541] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:23.166489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.166934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.167221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.167509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.168382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=851cf4bd4efd448f8fc04563775de1ac) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:23.169033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:23.171119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:23.178990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 263/542] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:22 2022] GET /identity/v3/role_assignments?user.id=851cf4bd4efd448f8fc04563775de1ac&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:23.324135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f6ea-9076-4825-bd0b-958050398395 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.353441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.378391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.379282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.379715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.380145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.382185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=fb22d91ffd224b6aa9093a440d5e4925) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:23.383791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:23.385440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:23.392643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 280/543] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/role_assignments?user.id=fb22d91ffd224b6aa9093a440d5e4925&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:23.546744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.674773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:23.710180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be29ede-f018-4db0-bdab-04e85fcee080 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.755157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.755710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.755990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.756263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.767810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:23.772554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:23.773465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:23.774321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 264/544] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:23.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:23.942930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f27e3e-4f15-4499-8891-2af85021cdbe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:23.995218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:23.996152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:23.996618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:23.997795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:23.998682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:23.999172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.000771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.017050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 281/545] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:24.022900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.197641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3125e1-b318-476b-82fa-b1e03747b1ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.241445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.242136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.242422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.242702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.244914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.255440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.256921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-93e7f0ec-45ad-420d-a74b-1616dc4177bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:24.282352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:24.283307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:24.284675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:24.284675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:24.284675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:24.286435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 265/546] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:24.292236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.383033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.456518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac0ebb-2075-4b40-90d6-d66a0e50e7a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.460690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b87f-3b2c-4283-9660-9df497aab677 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.460985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b87f-3b2c-4283-9660-9df497aab677 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.461259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123b87f-3b2c-4283-9660-9df497aab677 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.471166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 266/547] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:24.472524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.472815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.473093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.508185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.509166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.509674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.510196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.511066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:24.511574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.513707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.517710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:24.518948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:24.533024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 282/548] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:24.534571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.663705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 267/549] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:24.700195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90a186-543c-4894-868c-b04041a7b097 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.709210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.740321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.741338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.741875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.742354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.758810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.760637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:24.775016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:24.775995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:24.777002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:24.784895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 283/550] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:24.786696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.896956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9038c4-e8f5-49ac-b844-f27817f017ac None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:24.915911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:24.936014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:24.936954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:24.937467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:24.937979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:24.938922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:24.939435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:24.941073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:24.965256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 268/551] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:24.967473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:24.994870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.175917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560b7244-c43f-4946-a2db-56bc9d15bd31 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.179373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b524e-9f30-499d-93fd-e18dcc68116f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.250322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb302755-17dd-42e2-8aca-85b185d4a5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.250949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb302755-17dd-42e2-8aca-85b185d4a5f4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.251236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb302755-17dd-42e2-8aca-85b185d4a5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.251518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb302755-17dd-42e2-8aca-85b185d4a5f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.252381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.253066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.253341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.253612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.276474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:25.279730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:25.280745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:25.286444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 269/552] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:25.288080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.316936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.334210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.396182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb302755-17dd-42e2-8aca-85b185d4a5f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.397669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb302755-17dd-42e2-8aca-85b185d4a5f4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.407903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3c565c-e4d7-4734-857f-3937315c02c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.453298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.453872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.454156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.454433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.455092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.455851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.457518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53adc1f4-2e9f-431f-b764-24f8027e84ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.468884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 270/553] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:25.477049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c72d-d55d-455f-9165-d8e6e4574990 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.477049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c72d-d55d-455f-9165-d8e6e4574990 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.477049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1c72d-d55d-455f-9165-d8e6e4574990 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.477753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 271/554] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:25.478758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c72d-d55d-455f-9165-d8e6e4574990 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.518897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 284/555] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:25.520260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.520614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.520982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.540228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:25.541702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:25.602616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1c72d-d55d-455f-9165-d8e6e4574990 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.606203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.622595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.632781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 285/556] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:25.634093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.655856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.659286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.659286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.659286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.677566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.679443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1ab1c0d7-f2f6-4518-9090-53e711dc31aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:25.707766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:25.708730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:25.709610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:25.709610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:25.709610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:25.709610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 272/557] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:25.710926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158a13-9447-46ed-8916-732219eef479 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.711204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158a13-9447-46ed-8916-732219eef479 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.711588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7158a13-9447-46ed-8916-732219eef479 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.713003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 273/558] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:25.718634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158a13-9447-46ed-8916-732219eef479 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.766622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f5961-7a69-4781-a207-b6b773b7f98b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.808379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.809010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.810029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.810235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.820797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.822337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.831754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7158a13-9447-46ed-8916-732219eef479 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-84b00d42-7cdb-4b8d-9f17-37f922d3837c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:25.838425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:25.839642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:25.841080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:25.841080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:25.841080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:25.841080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 286/559] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:25.842201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.842201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.842380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.860811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:25.862040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:25.874782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:25.875736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.876274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.876554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.877192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:25.877494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:25.879012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a6cce-0867-47a4-a7b3-5c891b92bed0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:25.897037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 274/560] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:25.898486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be19f-eb92-4ba9-8254-41dbe04611ce None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:25.898773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be19f-eb92-4ba9-8254-41dbe04611ce None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:25.899046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8be19f-eb92-4ba9-8254-41dbe04611ce None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:25.901849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 275/561] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:25.901849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8be19f-eb92-4ba9-8254-41dbe04611ce None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:25.909373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.917039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:25.934722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 287/562] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:25.935725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.027258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8be19f-eb92-4ba9-8254-41dbe04611ce None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.040984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c04edb3-3c5a-4c4f-bbf8-458bc289dd33 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.068465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.069087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.069450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.069715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.070315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.070629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.071970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0af324-3182-4a18-b85e-1c32953f2fdb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.080438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.081081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.081829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.082109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] PATH_INFO: `/v3/users/glance` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.088774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 276/563] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:26.088774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.088774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.088774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.095416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.096866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.105204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:26.107259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:26.109306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.110265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 288/564] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.111307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.155370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.180569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.193759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 277/565] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:26.193759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.234824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5057f2-5213-4b0e-84e9-49e0a4c0d03c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.269965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.270597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.270884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.271152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.282503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:26.287685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.288675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:26.289533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 289/566] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.290831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.308553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13995809-9cb0-41d5-a1ff-eeddb6ad3c41 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.356270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.357214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.357624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.358118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.368345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.370660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:26.385904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.386878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:26.388467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 278/567] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.388960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.431895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.437357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf3f55-f3ca-4185-ba62-d391b6e20107 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.457007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.481156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.482055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.482347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.482625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.483362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.488166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.488166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.503445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 290/568] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:26.503797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d5b64-adec-43ad-8c2e-d2ffdefc0b14 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.504268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.541748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.542404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.542692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.542975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.553950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:26.559929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.560856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:26.561680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 279/569] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.562400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.608495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2d26d-a765-4021-9dbf-9f999060363c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.642236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.642844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.643123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.643394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.644050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.644356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.645753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.653946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 291/570] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:26.654751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.664957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a230cc4-ae2b-4eab-9a22-f431cf69a2fe None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.675978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.683006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.705133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.705790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.706073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.708252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.708252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.708252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.709261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.723442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 280/571] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:26.724280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.772113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa38ac6-eff7-4df5-ac76-9b33262b6eab None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.806260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.806812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.807088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.807360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.817895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:26.823248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.824367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 292/572] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.825463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.828627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e9539-c2f5-486a-8edd-e1ad46e3ee56 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.864087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.864720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.865003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.865281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.876151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.877610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:26.892343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:26.893294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:26.894218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:26.894218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:26.894218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:26.894218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 281/573] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:26.894218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:26.908208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.929530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dacaa9-bfa5-461f-94c4-87cad02a931f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:26.930602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:26.964868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:26.965721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:26.966012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:26.966290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:26.966949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:26.967251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:26.968664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:26.976647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 293/574] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:26.977436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.006377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713a6c5-ccd9-4d15-afa8-609d2d15c3a6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.042156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.042782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.043070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.043343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.054010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.055466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:27.069870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:27.070793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:27.071690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:27.071690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:27.071690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:27.071690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 282/575] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:27.071690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.081923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ebd14d-1663-41ce-8679-25c539a920ec None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.116916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.117511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.117818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.118093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.118809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=6a7f47fa1a8e4902a1a68219366274c2) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.119118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.120555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.125884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 294/576] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:26 2022] GET /identity/v3/role_assignments?user.id=6a7f47fa1a8e4902a1a68219366274c2&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.126812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.144920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.161634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.182402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2211ca8-5e89-4160-aedd-b0f8a3901321 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.219268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.219900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.220186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.220467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.221148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.221459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.222927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.232503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 283/577] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.233601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.264524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91199e88-1780-446f-b4b6-ed13edee30a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.303587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.304296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.304649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.304990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.305748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.306179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.307630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.325646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 295/578] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.326636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.351272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9de01-ae4d-4734-a155-a268b0665fe3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.396576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.397601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.398371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.399642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.399642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.412156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.415724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.421000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:27.433334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:27.435023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:27.436183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:27.436183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:27.436183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:27.441565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 284/579] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:27.441565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.453842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa0f38f-426b-4057-a82d-d95079062312 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.500214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.500913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.501267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.501607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.502338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.502709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.504142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.517536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 296/580] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.519100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.566130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04f2fdd-f53c-4be5-8119-b0b9ea31d5ec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.603713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.604271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.604551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.604826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.619611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.622174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.634245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d406a2-f6a4-41dc-8229-92fde2b5d4e4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:27.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:27.638868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:27.639827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:27.639827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:27.639827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:27.639827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 285/581] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:27.640057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.665606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.678218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.678853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.679267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.679553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.687200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.688766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.697184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:27.702195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:27.703305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 297/582] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:27.709249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.760822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602a6f23-7e88-4570-b0c9-d5cd7cf642f6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.798029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.798639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.799044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.799322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.799957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.800260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.801755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.815292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 286/583] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.816110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.828206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4a4d6-4bce-4029-a6da-ab596546afb5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.867747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.868707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.869213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.869814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.881333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.882852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:27.895202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:27.896185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:27.897135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:27.901032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 298/584] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:27.901915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:27.927570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461e1457-b7c7-4dc8-9ac0-15b279bf5401 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:27.938396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.958497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:27.966720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:27.967396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:27.967679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:27.967956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:27.968604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:27.968912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:27.970360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:27.982550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 287/585] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:27.982550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.023550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c561991d-2ecd-476a-a843-68d3fb93248b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.064199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.065104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.065387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.065805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.066502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:28.066924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.068445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.084042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 299/586] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:28.085210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.106355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfff5b6-fee7-4232-9597-5d2ac70b7988 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.143366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.143935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.144224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.144505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.153293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.154776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:28.166344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:28.167427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 288/587] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:28.168876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.204708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.212227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf71785-2c49-4510-96ab-7e28ccd2423f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.216240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.248731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.249351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.249633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.249944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.250723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:28.250981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.252428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.272284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 300/588] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:28.273120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.291196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fef9a0-608d-4692-b868-391709e82ea6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.333376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.334056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.334339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.334613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.343528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.345100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:28.358326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:28.359988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:28.362218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 289/589] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:28.364903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.411145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d2e9d-acfe-4646-a26d-423b1c9ab268 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.448573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.449257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.449634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.450795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/d0f8fc283cce4eff8126074e2a2c39fa/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.462889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.470199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=d0f8fc283cce4eff8126074e2a2c39fa, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.472348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.477739 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ddafec-6ee2-4bd1-b6ea-cbeef5d5c4d2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.514284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.514892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.515179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.515459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.516199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:28.516511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.517970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.535329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 290/590] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:28.536162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.553758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 301/591] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:28 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/d0f8fc283cce4eff8126074e2a2c39fa/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:28.553758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.568252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.677838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91e093a-1d64-47a6-ac61-dd2355c486d0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.687779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372bffc8-c777-4499-9e64-892d498f0304 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.724848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.725468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.725910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.726069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.726809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:28.727440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.729046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.740139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.740796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.741082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.741359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.750084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 302/592] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:28.750718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.753770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.754613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.766046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:28.775391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:28.781505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:28.782484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:28.782484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:28.782484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:28.782484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 291/593] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:28.782484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:28.859357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:28.901932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0dd90-c26b-4d2f-8b8d-1a4e63f142e7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.924853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc912ed-dd0e-442b-adf0-7d22d9803b1e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:28.952103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.953363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.954175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.954682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.967454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:28.968105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:28.968398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:28.969092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:28.971577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.974064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.982043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:28.983960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:28.997120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:28.998351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:28.999252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:28.999252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:28.999252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:29.000431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:29.001336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:29.002258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:29.002258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:29.002258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:29.002258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 292/594] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:29.002870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.008519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 303/595] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:29.008519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.070882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.126901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2fb0d-e65b-4a0d-b20c-7f57128477be None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.134108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2f700-8b19-4a5c-99f6-d6fa4c8fe0bd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.157108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.166606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.167215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.169481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.169481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.169481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.169481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.170156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.170265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.170506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.170728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.170820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.171368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.171673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.173043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c540a-092d-4de1-bed7-219f92f7aba0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.190033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 293/596] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.190033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.190695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 304/597] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:29.191891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2fdd2-0ac1-4d80-82ce-02850de8660d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.192184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2fdd2-0ac1-4d80-82ce-02850de8660d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.192460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2fdd2-0ac1-4d80-82ce-02850de8660d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.194034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 305/598] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:29.194746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2fdd2-0ac1-4d80-82ce-02850de8660d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.303365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2c3a0-7bec-4766-8b5d-987daf14c8fb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.313945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2fdd2-0ac1-4d80-82ce-02850de8660d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.315217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.341806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.342516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.342746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.343022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.343771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.344078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.345443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.357493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.358172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.358455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.358731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.359850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 294/599] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.362590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.367200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.368674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b5b8e6a4-f54f-4912-97b8-e38c1a9e5b93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:29.380919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:29.381911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 306/600] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.387984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.388585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.403679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:29.404897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:29.446989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.480834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 307/601] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:29.482595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.503458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c96f5b-b0bf-476a-a50c-0a7d3b3db421 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.539919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.540654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.541015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.541386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.548903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.550477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:29.563522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:29.564457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:29.565358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:29.569425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 295/602] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:29.570379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.608487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.638999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89fed02-f5e6-440c-b88e-49ec4f54a0b6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.690049 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.690753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.691060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.691557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.699023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.701208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.701709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b0a82d-6132-4afd-b71e-c1cb89878e99 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:29.714905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:29.716563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:29.717639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:29.721259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 308/603] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:29.722091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.744776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.746070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.746577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.747760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.748934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.749394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.751013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.760273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.764029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 296/604] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.764880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.837652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54998974-525a-44de-bd6a-3680ef1a23d6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.866020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:29.877576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.878231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.878513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.878790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.879011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f204c-d5a9-4553-b45f-30815a4f4298 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:29.879482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.879789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.881225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.893369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 309/605] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.894192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.925477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:29.926124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:29.926546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:29.926682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:29.927427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:29.927664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:29.929089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:29.947519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 297/606] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:29.947519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:29.997762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.006398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a76b41-bfc3-40ac-b30e-4d14b859366c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.043448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.044467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.044761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.045030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/e1a623f1fc434449a76106e74756e35c/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.061014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5326e4f3-b1cd-4a7b-b155-4a7f104b818f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.061423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=e1a623f1fc434449a76106e74756e35c, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.062935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.097221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.097807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.098094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.098370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.110597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:30.115831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:30.116887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 298/607] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:30.117841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.132402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.177350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 310/608] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:29 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/e1a623f1fc434449a76106e74756e35c/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:30.190958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.234796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364dcc7-b3e0-4281-8866-34046e160375 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.280324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.281110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.281462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.281883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/fb22d91ffd224b6aa9093a440d5e4925/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.301224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=fb22d91ffd224b6aa9093a440d5e4925, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.302946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.379976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b372e-2f61-4dfa-afd4-4979673b1b25 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.414546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 299/609] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:30 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/fb22d91ffd224b6aa9093a440d5e4925/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:30.415603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.430964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9171317-6a81-4afd-88f3-3ee154351364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.433109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9171317-6a81-4afd-88f3-3ee154351364 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.434330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9171317-6a81-4afd-88f3-3ee154351364 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.434624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9171317-6a81-4afd-88f3-3ee154351364 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/851cf4bd4efd448f8fc04563775de1ac/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.468621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.475021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9171317-6a81-4afd-88f3-3ee154351364 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=851cf4bd4efd448f8fc04563775de1ac, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.476893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9171317-6a81-4afd-88f3-3ee154351364 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.530380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.579612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f9a4a0-967b-4a2f-8c96-a2b145e78379 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.593257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 311/610] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:30 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/851cf4bd4efd448f8fc04563775de1ac/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:30.620677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.622148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.622603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.622948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d04a6-f564-4b59-8853-07bdb780596b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.623173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.623280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d04a6-f564-4b59-8853-07bdb780596b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.623506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d04a6-f564-4b59-8853-07bdb780596b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.624238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:30.624754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.626350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.629291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 312/611] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:30.634048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.634048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.634048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.636650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 300/612] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:30.642447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.652337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:30.654095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:30.791142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 313/613] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:30.796934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd51c00e-7565-4c8d-baca-29a2f477489d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:30.822152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:30.835805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:30.836507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:30.836797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:30.837081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:30.849384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:30.850893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:30.866549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:30.867552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:30.868438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:30.868438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:30.868438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:30.868438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 301/614] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:30.874271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:30.989175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a367c-e170-4be9-86f0-1d1b6df69b09 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.019035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.031297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.031914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.032191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.032462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.033102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:31.033402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.034863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.047353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 314/615] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:31.052698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.109950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.215806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f62d2d-5463-4b91-bbc5-6a9909e521db None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.228821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670e9b6-3f19-4b67-8b79-8beea1d4862e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.251287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.251841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.252119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.252391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.263710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:31.268869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:31.269815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:31.270644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 315/616] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:31.275827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.289688 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.290420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.290713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.291109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.300691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.302334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:31.316250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:31.317230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:31.318326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:31.322135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.322796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 302/617] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:31.324896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.375273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.406150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15163146-2a73-4e0a-a732-bb95ed13c0be None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.474932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702c629-82bc-4f4d-90fc-df6c1250930b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.488089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.488721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.489006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.489286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.489995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:31.490304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.491692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.504165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 316/618] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:31.514725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.532253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.532873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.533151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.533424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.534208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:31.534513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.536531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.556378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 303/619] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:31.566930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.741337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3557e135-1d96-4f58-ad6d-0dc0154f993d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.747437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.758223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:31.769301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa38149-c3a3-46f3-9ca2-02b0c145ee17 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:31.819059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.819718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:31.819831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.819981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.820266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.820438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:31.820803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:31.821184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:31.834698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.836262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:31.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:31.855555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:31.857250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:31.857250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:31.857250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:31.857822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:31.858301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 317/620] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:31.859823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:31.863887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:31.892032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:31.893253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:31.894183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:31.894183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:31.894183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:31.903620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 304/621] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:31.909598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:31.984414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0087d29-3242-49f4-9f92-93d61ce16c82 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.031688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.032253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.032538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.032815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.033493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:32.033831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.035325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.056739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 318/622] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:32.062914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.084709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.173489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130480c-0503-4d62-9289-b9f721af8797 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.217198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.217846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.218135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.218418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.219067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:32.219376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.220747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.239978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401799df-05da-4e97-9d3b-b0f9a8d8f679 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.253951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 305/623] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:32.259754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.263612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.312041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.312690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.312975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.313255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.334581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.336065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:32.353952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:32.354981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 00:49:32.356907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:32.367504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 319/624] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:32.380789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.428348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d596a0-99d5-42ce-a666-119b32d21fa1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.468066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.471109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.471408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.471747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.479901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.482170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.489564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-57a9692a-3b27-43d0-b9b1-77f07dccfc3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:32.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:32.498007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:32.498932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:32.501006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 306/625] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:32.502476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ea7e2-8001-4bad-aa4d-b2b0b48f6495 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.502940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ea7e2-8001-4bad-aa4d-b2b0b48f6495 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.503425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ea7e2-8001-4bad-aa4d-b2b0b48f6495 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.509079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 307/626] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:32.510159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ea7e2-8001-4bad-aa4d-b2b0b48f6495 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.517050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a793-c1b7-4716-af44-4358d13e3224 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.561616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.562481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.562698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.562962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.563700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:32.564009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.565755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4caba5-680c-434e-b84d-8ae82a38f893 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.585064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.593840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 320/627] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:32.593840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.593840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.593840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.614385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:32.615992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:32.636023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ea7e2-8001-4bad-aa4d-b2b0b48f6495 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.678496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.679475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.679946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.680534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.681612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:32.682162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.683949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.698994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 308/628] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:32.700817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.705661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 321/629] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:32.706980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:32.847692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.860892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6ea1b-9df8-45c5-9b2b-0a09bd3716cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.894103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:32.904371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e40b4-1442-4ba5-90b0-20ee88f0d90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.905020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e40b4-1442-4ba5-90b0-20ee88f0d90b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.905308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e40b4-1442-4ba5-90b0-20ee88f0d90b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.905587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e40b4-1442-4ba5-90b0-20ee88f0d90b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.921492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e146-e738-4e19-9d5a-ace128d5db60 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:32.978203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130ba85b-0431-45f2-990e-0425011b70f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:32.982412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ba85b-0431-45f2-990e-0425011b70f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:32.982412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ba85b-0431-45f2-990e-0425011b70f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:32.982412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130ba85b-0431-45f2-990e-0425011b70f1 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/6a7f47fa1a8e4902a1a68219366274c2/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:32.987471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e40b4-1442-4ba5-90b0-20ee88f0d90b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.988941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e40b4-1442-4ba5-90b0-20ee88f0d90b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:32.995737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ba85b-0431-45f2-990e-0425011b70f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=6a7f47fa1a8e4902a1a68219366274c2, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:32.997766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130ba85b-0431-45f2-990e-0425011b70f1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.080029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 322/630] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:32 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/6a7f47fa1a8e4902a1a68219366274c2/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:33.081501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196a49-6652-45b3-921e-d6f496f27882 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.081830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196a49-6652-45b3-921e-d6f496f27882 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.082116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d196a49-6652-45b3-921e-d6f496f27882 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.083597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 323/631] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:33.087564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1f4cd-f326-42ef-8186-3926d6cece8d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.088222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1f4cd-f326-42ef-8186-3926d6cece8d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.088506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b1f4cd-f326-42ef-8186-3926d6cece8d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.095672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 324/632] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:33.095672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1f4cd-f326-42ef-8186-3926d6cece8d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.113889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 309/633] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:33.113889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400942-704f-4873-8eac-af223dd5179a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.113889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400942-704f-4873-8eac-af223dd5179a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.113889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85400942-704f-4873-8eac-af223dd5179a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.134904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-85400942-704f-4873-8eac-af223dd5179a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:33.136694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-85400942-704f-4873-8eac-af223dd5179a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:33.163267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.220607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 310/634] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:33.222251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.222684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.223131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.234756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b1f4cd-f326-42ef-8186-3926d6cece8d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.249819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:33.251034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:33.252786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.276079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.276772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.277053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.277323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.289001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:33.294706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:33.295660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:33.296514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 325/635] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:33.297256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.333184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 311/636] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:33.334505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.441046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.483954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100af51-512f-4aa4-882e-71e19aa78dfd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.509815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910748d9-5656-4710-9a66-a1706530c900 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.528847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.529616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.530137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.530520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.531403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:33.531815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.533292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.542936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 312/637] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:33.544079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.563849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.564540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.564819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.565513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.580813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.582578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:33.604524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:33.606468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:33.608518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:33.608518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:33.608518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:33.608518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 326/638] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:33.608518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.643295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.669888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c942a0-3c67-46d9-b26b-3e31f84a357a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.708859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.709413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.709696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.709993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.721551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:33.726727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:33.728800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:33.732821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d7ca01-6ae8-49c0-b8aa-4a07f2a4d105 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.732983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 313/639] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:33.733753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.742502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.770899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.771546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.771825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.772101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.783622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.785084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:33.800159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:33.801333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:33.802307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:33.802307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:33.802307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:33.802307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 327/640] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:33.802546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.856630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff0f182-e7b2-4a49-a7ba-afc70dbf3e92 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.886486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:33.895307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.896427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.896717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.903691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.903691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:33.903691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:33.903691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:33.909154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd15b1-bae8-4659-a92a-cce316e87ad4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:33.916703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 314/641] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:33.917892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.947614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:33.948240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:33.948550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:33.948798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:33.961240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:33.967292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:33.968508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:33.969614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:33.973229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 328/642] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:33.974108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:33.994468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.036423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c96d2e-4d0d-4c32-887d-bc6422fe61db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.079050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.079662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.079941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.080107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91215aff-e707-42b0-9ad1-f84bc7ab7341 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.080208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.080859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.081162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.083090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.096626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 315/643] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.096626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.120320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.120935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.121223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.122148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.122790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.123097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.124470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.132163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.141671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 329/644] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.141671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.225502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd64441-f18b-43e0-88fc-57192f463115 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.245514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf827e-957d-4d2f-8721-6da7b5e731e8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.259375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.269476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.271014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.271014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.271014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.278847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.280382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.282512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.283203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.283487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.283770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.284424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.284729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.286130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:34.292676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:34.294139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:34.295194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 316/645] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:34.296223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.304091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 330/646] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.304091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.363708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.411966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe620658-015b-4e37-beba-a881d640f890 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.423646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede15ee9-775c-4d1a-aa87-0e1e235d8129 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.460912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.461551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.461865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.462145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.473758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.474054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.474629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.474916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.475196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.475293 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:34.487508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:34.488453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:34.489407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 331/647] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:34.490056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.492514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.494010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:34.509516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:34.510557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:34.512084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:34.512084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:34.512084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:34.512084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 317/648] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:34.512084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.557656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.611917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7931a0-628c-47ca-a874-79e92ef8d5c0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.627107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babaa3f2-7ffb-4e43-8d4c-ad99b0ff2347 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.637689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.654565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.655306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.655647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.658695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.658695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.658695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.658695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.668785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.669427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.669716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.670185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.672162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 332/649] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.674604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.689357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.691073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:34.706727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:34.707897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:34.708948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:34.708948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:34.708948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:34.708948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 318/650] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:34.708948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.795258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15aa39-5594-4890-9ec0-aa45dd59aebf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.813920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.823379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b5f16-6240-4e9b-b987-c29b01050766 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.834147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.834763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.835048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.835508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.836038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.836341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.837788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.863350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 333/651] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.864273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.865238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:34.865894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:34.866182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:34.866462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:34.867110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:34.867421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:34.868800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:34.881868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 319/652] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:34.887154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:34.906725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:34.976744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0979e53-fc14-424b-b190-135b7b48199c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:34.993747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb727dc0-6f74-4d30-b892-b106ca93bff7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.019797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.020407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.020691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.020962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.021673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=d0f8fc283cce4eff8126074e2a2c39fa) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:35.022018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.023442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.037755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 334/653] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/role_assignments?user.id=d0f8fc283cce4eff8126074e2a2c39fa&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:35.037971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.038082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.038197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.038197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.038197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.038197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:35.038197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.038197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.052551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 320/654] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:35.053486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.071909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.175959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf135b-cc93-42ed-b2bf-61081168e387 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.177653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.189370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c2e1d-f2f2-43b0-b364-8b7708019683 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.221143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52c64-3cd4-402e-9bc7-80b641c5d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.222062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52c64-3cd4-402e-9bc7-80b641c5d822 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.222342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52c64-3cd4-402e-9bc7-80b641c5d822 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.222613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52c64-3cd4-402e-9bc7-80b641c5d822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.228441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.229083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.229370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.229653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.239169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.240624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:35.253998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:35.254974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 321/655] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:35.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.299190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c52c64-3cd4-402e-9bc7-80b641c5d822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.300655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c52c64-3cd4-402e-9bc7-80b641c5d822 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.343242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.373574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0a8dc-f828-419f-b68f-4e3ed3cc7c35 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.411582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.412231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.412518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.412798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.415000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 335/656] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:35.415211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb523a-3b95-41bb-9f50-d07fc792a101 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.415559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb523a-3b95-41bb-9f50-d07fc792a101 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.415769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb523a-3b95-41bb-9f50-d07fc792a101 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.417077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 336/657] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:35.417810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb523a-3b95-41bb-9f50-d07fc792a101 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.424539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.430598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.432093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:35.444673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:35.445926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 322/658] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:35.447098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.542326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb523a-3b95-41bb-9f50-d07fc792a101 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.573552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38a186-efb6-461c-bca6-0b0640528447 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.580942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.581477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.581779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.582056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.582784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:35.583089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.584535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2eb724-8605-4c3d-aa33-6c3da0efd676 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.603412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 337/659] 10.222.0.52 () {56 vars in 1137 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:35.604593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.604880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.605153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.616853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.622684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.623463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.623971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.624254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.624550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:35.625874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:35.635585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.637674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:35.652930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:35.654563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:35.656096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:35.656096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:35.656096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:35.656096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 323/660] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:35.657140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.699985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.713774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 338/661] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:35.713774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.796406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15d82b7-ef52-4063-87e7-ddcaf5a63ebf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.840787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b0206-0653-453c-8b92-d84a1ef6c1fb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:35.845909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.846717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.847147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.847569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.848619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:35.849092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.850700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.870944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 324/662] 10.222.0.52 () {56 vars in 1139 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:35.872887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:35.890839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:35.891577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:35.892100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:35.892660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:35.922054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.959565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:35.971211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:35.972841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:35.991108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373fb9b1-7f4d-40ac-ba8d-71f4260cf0ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.033853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.034491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.034744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.035017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.035668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.035982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.037350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.050029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 325/663] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.050863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.084768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 339/664] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:36.085701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.168839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e6485-336e-4304-884d-1a639b949983 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.170822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.197444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.207701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.208253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.208540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.209088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.212997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eabea3-511d-44c2-8c86-f979fbe12847 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.257472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.258160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.258437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.258711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.270858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:36.275875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.276979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:36.278264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 340/665] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:36.283207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.299044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.301010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.395542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3dfb21-a84c-4917-9a78-795e4ed4da58 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.416452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 326/666] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:36.417378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.420535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.437971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.438656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.438940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.439212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.446386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.447921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:36.459813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.460939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 341/667] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:36.467301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.484210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.545347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b896850-cf74-4eaf-87e5-fe876a01fe69 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.597102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.597773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.598056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.598343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.600967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac2675b-54b9-4fbb-b508-d18d13b808b0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.612085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.613568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:36.629038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.630252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:36.631318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:36.631318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:36.631318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:36.634513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 327/668] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:36.635332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.644833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.645820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.646276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.646705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.647637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.648093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.649804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.658972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 342/669] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.660015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.699058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.737683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.749830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05c6a9-097a-4f7c-9fc2-17299f598dbb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.769172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca210d-77aa-4cc6-b87a-0d1fbd3a31a6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.789563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.790719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.791009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.791567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.803949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.805395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.805519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.806308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.806709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.807091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.807888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.808203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.809759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:36.820298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:36.821268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:36.822210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:36.822210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:36.822210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:36.822210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 328/670] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:36.822210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.833632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 343/671] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.833632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.934012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962f0e6-3389-4f99-9114-11cf9c57a937 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.939940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd44847e-20d4-4d17-be38-3e2bfd1f205b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:36.942875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.973219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:36.975051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.975678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.975986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.976248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.976883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:36.977190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.978604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:36.979465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:36.980307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:36.980853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:36.981318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:36.992888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 329/672] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:36.993083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:36.994145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:36.994678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:37.016616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.017861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:37.018759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:37.018759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:37.018759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:37.018759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 344/673] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:37.018997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.107371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caa104-96e0-4790-a8c0-7958a9948621 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.129521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9b4b84-ca78-49c1-bd02-5ab9db5e7a21 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.142090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.142711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.142999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.143281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.143927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.144238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.145605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.157939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 330/674] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.158778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.164033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.164645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.164921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.165216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.165970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=e1a623f1fc434449a76106e74756e35c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.166277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.167705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.173902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 345/675] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/role_assignments?user.id=e1a623f1fc434449a76106e74756e35c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.174699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.188119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.238221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.277252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d451da-384e-410c-9f25-0842620ceb6c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.299642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d660148-f912-492c-8fc9-cd535df5e6bc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.323024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.330417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.330417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.330417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.332281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.334599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.344100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.345409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.345409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.345409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.346085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.347399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.347844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-15fdedf4-97d0-4318-ba97-9033096131bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:37.351474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.352391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:37.353996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 331/676] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:37.354480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced0c49-dd2d-4ec2-8174-0f7fce95c5a1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.354480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced0c49-dd2d-4ec2-8174-0f7fce95c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.354918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced0c49-dd2d-4ec2-8174-0f7fce95c5a1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.356723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 332/677] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:37.358218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced0c49-dd2d-4ec2-8174-0f7fce95c5a1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.363305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 346/678] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.364160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.459200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.485012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced0c49-dd2d-4ec2-8174-0f7fce95c5a1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.489445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6bff5c-a7be-44c6-a530-216efd6f5f05 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.501557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.520303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.520944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.521223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.521495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.528310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.529697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.531476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.532435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.532893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.533418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.534347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.534847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.536684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f57e1e6b-94c6-43fe-9776-c2da1ce2898d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:37.541024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.542078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:37.543950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:37.547432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 333/679] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:37.548651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.548942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.549218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.558513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 347/680] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.558513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.573334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:37.575255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:37.649587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 334/681] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:37.650393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.678920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206714d-b936-4760-bbc5-e9433dc18e60 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.719167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.720087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.720372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.720779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.729161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.731083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:37.743525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:37.744731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:37.745933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:37.747938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 348/682] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:37.748987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.760916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.786799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6857b857-c06a-4eda-9103-a7673ddeeec2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.788188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:37.827285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.828064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.828541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.828922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.829810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.830242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.831882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.847788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 335/683] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.849660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.888021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d540e9af-c1b8-4c83-8c8e-df04ceb2ed05 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:37.924393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:37.925254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:37.925684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:37.926158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:37.927015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=fb22d91ffd224b6aa9093a440d5e4925) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:37.927491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:37.929116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:37.942151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 349/684] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/role_assignments?user.id=fb22d91ffd224b6aa9093a440d5e4925&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:37.943374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:37.979674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081eb6ef-b606-426f-bb83-bff45cfa921a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.030494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.031231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.031621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.032004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.032868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.033290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.034913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.038465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.056049 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.060551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 336/685] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.061897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.065428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91842de-5272-4e8b-94fc-38b83d9269af None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.102996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.103618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.103906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.104186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.105821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=851cf4bd4efd448f8fc04563775de1ac) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.105821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.107122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.114097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 350/686] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:37 2022] GET /identity/v3/role_assignments?user.id=851cf4bd4efd448f8fc04563775de1ac&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.118810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.215095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c41b4-538e-4534-aae7-85eb61b9bfcf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.256242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9726d-3062-404e-880c-526ad26920e0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.266513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.267309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.267598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.267877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.280891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.282338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.298055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.298740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.299028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.299364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:38.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:38.303009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:38.303923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:38.303923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:38.303923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:38.304697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 337/687] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:38.313712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.339388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.379369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.393749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.396604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.459090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f3903-e515-493b-a365-e490e81a2721 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.513034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.513659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.514733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.514733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.514984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.515297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.516658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.533987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 338/688] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.538011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.550070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 351/689] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:38.555554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.618985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.664888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61911b61-91ab-4a9c-9f1a-9d7634b0f545 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.665621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad7610-8292-4fea-94cf-40109a3f3327 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.673863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.703570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.704221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.704511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.704788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.705207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.706077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.706642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.707471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.708794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.710461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.716677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.718181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:38.734789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:38.735805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:38.737344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:38.737344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:38.737344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:38.737344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 352/690] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:38.741882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:38.801372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 339/691] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:49:38 2022] POST /identity/v3/users => generated 254 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:38.858085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291b9ce-a26a-4bc7-8af5-6c0dee9010af None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:38.870582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:38.897930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:38.898557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:38.898849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:38.899181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:38.899934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:38.900243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:38.901671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:38.916670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 353/692] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:38.922773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.055057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a852862-bbef-47cb-9e88-dfe18ac7d58e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:39.092052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:39.092700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.092984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.093264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.100644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.102237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:39.114877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:39.115806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:39.116678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 340/693] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:39.122845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.229907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97d833-a055-485e-bcf9-47c2ecac2603 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:39.267633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:39.268318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.268602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.268881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.269638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:39.269984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.271443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8b9e33-354c-405d-b255-7a67217cc194 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.284940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 354/694] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:39.290922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.319090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.403597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ed7b2d-c927-45fa-badb-4c1d55e12516 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:39.440105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:39.440722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.441007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.441288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.442052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=6a7f47fa1a8e4902a1a68219366274c2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:39.442366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:39.443800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:39.448068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e224b7-4bb2-49b3-a511-fbf2031423bd None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.448903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e224b7-4bb2-49b3-a511-fbf2031423bd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.449231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e224b7-4bb2-49b3-a511-fbf2031423bd None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.451181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 355/695] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:39.451717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 341/696] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:39 2022] GET /identity/v3/role_assignments?user.id=6a7f47fa1a8e4902a1a68219366274c2&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:39.456719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efc222-3d18-4792-b23e-684431742c85 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:39.457009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efc222-3d18-4792-b23e-684431742c85 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:39.457294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efc222-3d18-4792-b23e-684431742c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:39.497078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-89efc222-3d18-4792-b23e-684431742c85 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:39.498362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-89efc222-3d18-4792-b23e-684431742c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:39.538090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.598710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 356/697] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:39.890238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:39.983215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:39.999644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d931a4-a771-4db7-b2c0-1012ba328378 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.042101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.042762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.043647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.043926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.059411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:40.064511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:40.065448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:40.066310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:40.068654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 342/698] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:40.074782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efc222-3d18-4792-b23e-684431742c85 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.196238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efc222-3d18-4792-b23e-684431742c85 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.241328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.242361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.242866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.243335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.244195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:40.244693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.246832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e0a21-96de-431f-b548-6e76a87a930d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.256765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 357/699] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:40.262825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.268821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d288b-fd4d-4d76-bbfe-88689d23e26d None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.269375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d288b-fd4d-4d76-bbfe-88689d23e26d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.269889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d288b-fd4d-4d76-bbfe-88689d23e26d None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.271756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 358/700] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:40.277120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.277700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.278208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.297935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:40.299475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:40.315981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.376719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 359/701] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 00:49:40.447883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a65226-b1e7-4248-a4e7-7ddbc0622e73 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.509216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.509891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.510181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.510502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.535782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.537262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.550029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:40.556340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:40.557359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:40.558292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:40.558292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:40.558292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:40.562468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 343/702] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:40.565247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.688634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0753-4d0c-4901-8089-b0fd6a1388f0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.706928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.726839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.727979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.728477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.728943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.729921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:40.731001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.732814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.747687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 360/703] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:40.756778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.788009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:40.902059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de6899-799b-4d17-8859-b6e793ff0967 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.936031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0df9dde-f54f-47e1-bc41-da502201fcac None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:40.954785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.956101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.956590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.957124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.957946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.960398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:40.974115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:40.974748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.975030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.975306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ff6d5492-7c2c-4889-8f22-e2fde9bdb981 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:40.977759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:40.978821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:40.979684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:40.982916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 344/704] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:40.984448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6daf5c-84e1-45ef-b7e7-95b7a0167d7e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:40.984916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6daf5c-84e1-45ef-b7e7-95b7a0167d7e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:40.985371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6daf5c-84e1-45ef-b7e7-95b7a0167d7e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:40.987132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 345/705] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:40.989620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6daf5c-84e1-45ef-b7e7-95b7a0167d7e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:40.991019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:40.992513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-8ddb4ad4-7d63-474a-b0b8-fded6bbc6bb9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:41.006791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:41.007862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:41.008843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 361/706] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:41.009426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.009692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.009966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.029193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:41.030667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:41.067362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.103682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.105165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 362/707] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:41.106525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.115250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6daf5c-84e1-45ef-b7e7-95b7a0167d7e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.164398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.165476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.171397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.171397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.171397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.171397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.171397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.175689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 346/708] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:41.181514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.291689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6178d-4e91-4c27-b0f5-80866ace85da None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.292510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681ba71d-af1d-4a2b-b5a9-47bcbaff5f4d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.331046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.331994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.332442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.333204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.333987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.334495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.336057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.338112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.344536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 347/709] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:41.378623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.379271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.379560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.379848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.380610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.380921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.382481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.425099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.466516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 363/710] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:41.473126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.560456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.587321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a25e1a-fd8c-41a6-93d7-b32cc41e1784 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.618744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac164b4-fc0a-45df-b20d-0d20a9809cc0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.628910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.629567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.629894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.630175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.630764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.632140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:41.642248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:41.643279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:41.644105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 348/711] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:41.649354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.707647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f85391-4dd3-40e5-b96f-2d50ad99e983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.708289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85391-4dd3-40e5-b96f-2d50ad99e983 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.708577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85391-4dd3-40e5-b96f-2d50ad99e983 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.709288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f85391-4dd3-40e5-b96f-2d50ad99e983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.743251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.802705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967c0b2-8555-452f-a0ca-c213d187a54d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:41.827834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85391-4dd3-40e5-b96f-2d50ad99e983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.830289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f85391-4dd3-40e5-b96f-2d50ad99e983 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.841621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:41.842285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.842572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.842877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.843600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] RBAC: Adding query filter params (name=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:41.843914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:41.845481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a8fae-96d7-41cc-a70d-4c14cd25662f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:41.851179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 349/712] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:41.855479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd749f-460b-4da9-88b9-2db4e872953e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.855479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd749f-460b-4da9-88b9-2db4e872953e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.855479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd749f-460b-4da9-88b9-2db4e872953e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.855479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 350/713] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:41.856447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd749f-460b-4da9-88b9-2db4e872953e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:41.905579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:41.969898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 364/714] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 00:49:41.971603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e323c-5b19-4eac-acb8-d4ad1f61ad2f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:41.972097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e323c-5b19-4eac-acb8-d4ad1f61ad2f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:41.973305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5e323c-5b19-4eac-acb8-d4ad1f61ad2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:41.981290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd749f-460b-4da9-88b9-2db4e872953e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.001465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-bc5e323c-5b19-4eac-acb8-d4ad1f61ad2f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:42.003795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.004530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bc5e323c-5b19-4eac-acb8-d4ad1f61ad2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:42.023417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.025464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.025464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.025464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.025996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:42.026401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.028135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.038956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 351/715] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:42.039946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.114035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 365/716] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:42.122562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c2e3c-cab1-4873-b629-d92afa8ca6ab None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.122917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c2e3c-cab1-4873-b629-d92afa8ca6ab None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.123134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c2e3c-cab1-4873-b629-d92afa8ca6ab None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.125066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 366/717] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:42.131356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.132079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.132079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.162205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:42.163566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:42.225955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d921d55-a504-422e-9eb2-ccd6f73d4633 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.273090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.274260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.274754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.275345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.276986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 367/718] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:42.287302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.289067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:42.305168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:42.306236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:42.307140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:42.307140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:42.307140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:42.314191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 352/719] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:42.320015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.349805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.442918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.459884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d5f889-9223-4376-b5f5-455d4b1930b5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.519466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.520093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.520378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.520650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.521400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:42.521707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.523209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.525214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.547493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 368/720] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:42.561410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.612207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5d90-a670-4e2c-a300-91cc8ba2e662 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.662650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.663881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.664809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.665245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.665871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.680669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:42.686449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca9c91d-87bf-4cd2-a82a-819b727219a6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:42.688357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:42.689504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:42.690564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:42.694069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 353/721] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:42.695261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.736738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.737634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.738086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.738516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.749031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.751313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:42.767374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:42.768327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 369/722] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:42.774313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.813436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.820269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304547e-a81e-445d-86c1-d3307aa651a4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.862037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.862691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.862975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.863253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.876119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:42.881014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:42.881978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 354/723] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:42.883458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:42.926216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade28d6-1882-464d-bc12-5f078568abfa None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:42.974605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:42.975556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:42.976046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:42.977066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:42.978073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:42.979054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:42.981997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:42.989563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:42.995810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 370/724] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:42.997516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.005713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e9c08-5a47-46e1-a291-e06ec0211844 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.044837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.045670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.046056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.046405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.047179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:43.047558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.049212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.068099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 355/725] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:43.068099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.086207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.136747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4561dd-8e6f-4244-a4da-dc32ca2378b6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.180315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee56d98-2fc5-41b7-8e93-fee239a405aa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.182327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.184747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.185223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.185653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.186505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:43.186987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.188539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.200514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 371/726] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:43.202038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.220941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.221786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.222220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.222596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.234446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.236071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:43.252677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:43.253938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:43.255050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:43.255050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:43.255050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:43.256387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 356/727] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:43.257409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.287750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.332854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.341951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168307d9-9d14-4472-8fbe-32d39029592a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.380429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.381370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.381770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.382155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.383008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=e1a623f1fc434449a76106e74756e35c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:43.383384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.385096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.386258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f94f24-71de-4268-9a5d-18f5ab5eeeb9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.397087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 372/728] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/role_assignments?user.id=e1a623f1fc434449a76106e74756e35c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:43.397087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.468785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.469507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.469964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.470247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.496078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.498086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:43.549137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:43.550230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:43.552397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:43.552397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:43.552397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:43.554331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 357/729] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:43.562109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.585324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bd6baf-78da-4633-b45a-c477a39e0337 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.614857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.624862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.625837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.625837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.626081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.626787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:43.627094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.628744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f8060-f8d0-4cb7-94b5-c67857ae5a93 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.643373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 373/730] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:43.644761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1731-02d2-493d-8c7b-debf03a93536 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.645055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1731-02d2-493d-8c7b-debf03a93536 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.645332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a1731-02d2-493d-8c7b-debf03a93536 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.650695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 374/731] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:43.650695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1731-02d2-493d-8c7b-debf03a93536 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.682202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3279051-5390-48c6-967f-4595a66b3761 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.702390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.720651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.721548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.722014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.722532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.723348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:43.723806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.725393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d96935b-896f-4478-99f0-67ae5a60304f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.739823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 358/732] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:43.741435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.741915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.742342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.761226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:43.763261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:43.769613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a1731-02d2-493d-8c7b-debf03a93536 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:43.810228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:43.810872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:43.811211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:43.811492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:43.819979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:43.821506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:43.835858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:43.837047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:43.838106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 375/733] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:43.838703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.845910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 359/734] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:43.847122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:43.897871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:43.957900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a19bd1-c490-47b4-a338-3a6b3f61b44c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.021936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.023758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.024043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.024317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.038703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.041960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:44.064487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:44.065513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:44.066512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 376/735] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:44.071064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.096541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106c98f-0751-4e23-93ed-6ea07c22a506 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.116440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.157054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.157696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.158116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.158450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.159196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:44.159502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.161078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.188705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 360/736] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:44.189667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.205870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f81bd-bcbd-402a-8add-1aee804b9df5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.213995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.244402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.245431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.246524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.247024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.247957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:44.248638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.250669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.270692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 377/737] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:44.271881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.299575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8ce60-8ba1-40e1-ad15-0e79fc1cd6b1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.338970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.339616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.339912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.340343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.340833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.342241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.384219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c0f4d-a8d6-47d0-a47a-31c3716bf5e2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.419952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a9c68-8c45-4779-b57d-38742b365cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.420559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9c68-8c45-4779-b57d-38742b365cbd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.420835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9c68-8c45-4779-b57d-38742b365cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.421103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a9c68-8c45-4779-b57d-38742b365cbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.456381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.458661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 361/738] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:44 2022] POST /identity/v3/services => generated 251 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:44.459331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.508009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9c68-8c45-4779-b57d-38742b365cbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.508211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.508291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a9c68-8c45-4779-b57d-38742b365cbd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.608513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca89f4c-e8d7-4030-802b-5297d57127b2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.625296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 378/739] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:44.626713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba7e30-66b8-4b04-8dcf-0826162ca374 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.627005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba7e30-66b8-4b04-8dcf-0826162ca374 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.627278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba7e30-66b8-4b04-8dcf-0826162ca374 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.632835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 379/740] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:44.633922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba7e30-66b8-4b04-8dcf-0826162ca374 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.659503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.660135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.661232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.661232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.722782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.764803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.765999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba7e30-66b8-4b04-8dcf-0826162ca374 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:44.766346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.795908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:44.805050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:44.805695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.806127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.806445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.818332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:44.819839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-eafde784-3b88-419d-8eba-03bf4397a204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:44.836171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:44.837174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:44.838482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:44.838482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:44.838482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:44.841865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 380/741] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:44.843200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:44.843493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:44.843775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:44.863995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:44.865173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:44.903989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 362/742] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 00:49:44.905155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:44.950043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 381/743] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 00:49:44.951234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.003122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.048109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b128364-b27b-4029-a785-4a75b36d0031 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.063467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6022acfc-71a9-4aff-a4e5-f5ca3d8d7d17 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.079644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.092489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.093447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.093961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.094459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.095334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:45.095834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.097511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.107919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.109037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.109389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.109713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.113193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 363/744] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.127542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.133721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.135249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:45.166209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:45.167214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:45.168258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:45.168258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:45.168258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:45.171016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 382/745] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:45.178774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.327889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7aa733-84d2-43d9-98fa-52bdf4de61ea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.350765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.355257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95269d23-7b94-470c-8f32-a6c21c0db4ac None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.367681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.368644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.369068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.369488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.377493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.379351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:45.391870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:45.393278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 364/746] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:45.400598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.401153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.401153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.401153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.401153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.401777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:45.401777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.403156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.417718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 383/747] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.419045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.455620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.528651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac80e2-31b9-4493-a0b9-4ff59eb034d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.539561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a53b0-7427-44fb-b3e8-1c2309e9c481 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.568116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d349d2ca-e03e-4751-9044-8acffb3be42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.569167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349d2ca-e03e-4751-9044-8acffb3be42f None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.569647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349d2ca-e03e-4751-9044-8acffb3be42f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.570156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d349d2ca-e03e-4751-9044-8acffb3be42f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.571009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349d2ca-e03e-4751-9044-8acffb3be42f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.572608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d349d2ca-e03e-4751-9044-8acffb3be42f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.580506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.581130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.581414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.581693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.582410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:45.582719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.584154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.596293 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 384/748] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.597119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.619955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.675061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 365/749] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:49:45 2022] POST /identity/v3/services => generated 240 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:45.676236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5d0b5-a907-470c-b1e9-b746d23b0c6c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.676529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5d0b5-a907-470c-b1e9-b746d23b0c6c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.676803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5d0b5-a907-470c-b1e9-b746d23b0c6c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.681813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 366/750] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:45.681813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5d0b5-a907-470c-b1e9-b746d23b0c6c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.714310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3263a55-128f-418e-a878-f1bbf815dd60 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.762943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.763718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.764087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.764430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.772601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.774447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-075ae434-09fb-4155-8ad0-578dcbd269ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:45.786672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:45.787654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:45.788579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 385/751] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:45.790040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.790391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.790735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.810806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:45.812117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:45.831408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5d0b5-a907-470c-b1e9-b746d23b0c6c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:45.834217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:45.880445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:45.880992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:45.881274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:45.881552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:45.882336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=3002491f2c61493890d2e4272e9ba522) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:45.882879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:45.884644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:45.895569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 367/752] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/role_assignments?user.id=3002491f2c61493890d2e4272e9ba522&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:45.895569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:45.909198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 386/753] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:46.016911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35837c25-e00f-461f-a0eb-485e312842fa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:46.054312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:46.055000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.055350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.055546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.056303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:46.056615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:46.058152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:46.073303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 368/754] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:46.080289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:46.104983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:46.209592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46f3ccd-967f-419b-bb8e-0baafe294e37 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:46.247249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:46.251135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:46.251770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.252335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.252615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.253350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=6a7f47fa1a8e4902a1a68219366274c2) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:46.253658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:46.255128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:46.267996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 387/755] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:46 2022] GET /identity/v3/role_assignments?user.id=6a7f47fa1a8e4902a1a68219366274c2&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:46.291552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:46.376489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff557445-5f7b-4b4f-bb84-dfd251fcf0ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:46.429672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:46.430338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.430616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.430889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.449344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:46.454268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:46.455204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:46.456449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 369/756] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:46.461432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:46.548193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:46.590420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e9a4fb-0bc3-4159-8d4e-b19beebc8aaa None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:46.628479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:46.629383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.629827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.630243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.631040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:46.631575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:46.633092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:46.642572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 388/757] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:46.648884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:46.759273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:46.775484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c4dc17-6e36-46d4-8310-70d2b32735b2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:46.812218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:46.812779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.813059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.813341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.825233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:46.826733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5ba7724c-97fd-4c2b-8e7f-c53e8fa0c08f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:46.843084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:46.844525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:46.845427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:46.845427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:46.845427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:46.845427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 370/758] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:46.850350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:46.944495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592a94e-e615-4b8b-bd7c-2094d96c3feb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.944818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592a94e-e615-4b8b-bd7c-2094d96c3feb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.945046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592a94e-e615-4b8b-bd7c-2094d96c3feb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.946539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 371/759] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:46.958354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:46.958644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:46.958936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:46.962786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965078-a22c-4aa6-b089-bb449c577f4b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:46.986011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:46.987365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:47.002085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:47.010610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:47.011227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.011705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.012097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.012799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:47.013101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:47.018994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:47.044927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 389/760] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:47.050739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:47.127698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 372/761] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:47.217630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab294a2-952a-4585-80fb-90dfd0ea148a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:47.262348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:47.263039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.263321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.263595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.275549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:47.277076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:47.291608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:47.293176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:47.294578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 390/762] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:47.298940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:47.377525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:47.412080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988e88d-0575-4949-9d80-e75ddb8d0447 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:47.447746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:47.448457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.448737 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.449927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.450663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:47.450967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:47.452469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:47.472360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 373/763] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:47.478108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:47.556310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:47.626735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f68fbd-f02a-42c3-8770-bf0eb71468ee None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:47.631244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:47.681344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:47.681934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.682222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.682502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.702191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:47.703729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:47.730478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:47.731725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:47.733238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:47.733238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:47.733238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:47.733238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 391/764] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:47.737531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:47.745421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732649c3-97ac-407c-ac77-e5cb724cf4d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:47.783135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:47.783827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.784116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.784397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.784986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:47.786492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:47.796328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:47.797332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:47.798799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:47.801866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 374/765] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:47.802088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:47.803677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:47.864549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9085874-7247-489e-9bd4-9d2785ba18b6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:47.878285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:47.903115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:47.903736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:47.904069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:47.904377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:47.904988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:47.905382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:47.906888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:47.920699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 392/766] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:47.926240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.038567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1aefc-78d7-43df-832c-10fcaac86afc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.063751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1264d74e-1e7f-459f-ab2e-609c7f8bbe5b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.077986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.078671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.078959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.079705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.086896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.089039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:48.101541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:48.102483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:48.103836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 393/767] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:48.110215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.139099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.198459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.199825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.200332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.200672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.201313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:48.201623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.203029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.219759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 375/768] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:48.224971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.225689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dbf2cc-639a-4631-85d0-26c3c695c7cf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.307467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.308626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.309462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.309984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.310903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:48.311416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.313073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c678d8aa-cc4d-4f28-ab0e-e5b901cc41a5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.321768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.331571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 394/769] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:48.333078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3fd3a-0a4d-4643-b040-e7d4ec6a648b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.333523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3fd3a-0a4d-4643-b040-e7d4ec6a648b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.333973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3fd3a-0a4d-4643-b040-e7d4ec6a648b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.339956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 395/770] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:48.341141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3fd3a-0a4d-4643-b040-e7d4ec6a648b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.353784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4f702-ce9f-4c53-b2e7-14efe2298f89 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.393612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.394373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.394754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.395237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.396081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:48.396586 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.398357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595f88-3248-46af-95e8-eab0a2c504b1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.405412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 376/771] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:48.406958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.407439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.407862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.428888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:48.430573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:48.438503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.457670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3fd3a-0a4d-4643-b040-e7d4ec6a648b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.499838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23dcbf9-6b40-4512-8359-bc8d5996be94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.501307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dcbf9-6b40-4512-8359-bc8d5996be94 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.501757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dcbf9-6b40-4512-8359-bc8d5996be94 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.502200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23dcbf9-6b40-4512-8359-bc8d5996be94 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/e1a623f1fc434449a76106e74756e35c/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.504547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dcbf9-6b40-4512-8359-bc8d5996be94 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=e1a623f1fc434449a76106e74756e35c, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.506147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23dcbf9-6b40-4512-8359-bc8d5996be94 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.514174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 377/772] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:48.515028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.572412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.629649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53f1da-654b-4e66-8bd5-8988f604f5ca None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.662453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 396/773] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:48 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/e1a623f1fc434449a76106e74756e35c/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:48.665349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ab160-b6eb-4808-a490-2e1f282fe4bb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.666076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ab160-b6eb-4808-a490-2e1f282fe4bb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.668957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ab160-b6eb-4808-a490-2e1f282fe4bb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.672573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 397/774] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:48.681830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4571f3-fcf7-4eab-aeec-183c227affbe None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.681830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4571f3-fcf7-4eab-aeec-183c227affbe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.681830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4571f3-fcf7-4eab-aeec-183c227affbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.692490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.693144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.693428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.693914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.694656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] RBAC: Adding query filter params (interface=public, service_id=af0e886914e94002b4d69e7d100159e5, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:48.694982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.696388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2250c37d-f7fc-4547-aeac-085ee4e83347 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.701942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ec4571f3-fcf7-4eab-aeec-183c227affbe None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:48.703447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ec4571f3-fcf7-4eab-aeec-183c227affbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:48.705284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 378/775] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/endpoints?service_id=af0e886914e94002b4d69e7d100159e5&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:48.706614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef9f2a-d020-4a96-b4c7-b20dd141992d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.706908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef9f2a-d020-4a96-b4c7-b20dd141992d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.707187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef9f2a-d020-4a96-b4c7-b20dd141992d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.712581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 379/776] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:48.712581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef9f2a-d020-4a96-b4c7-b20dd141992d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.811131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 398/777] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:49:48.813526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.813526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.813526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.837604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:48.839028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:48.857827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef9f2a-d020-4a96-b4c7-b20dd141992d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:48.882124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:48.898888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:48.899715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:48.900062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:48.913215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:48.913215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:48.913215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:48.913215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 380/778] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:48.913215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:48.936032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 399/779] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:49.134619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.137994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:49.175606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea7852-3d87-42c4-bcee-2b5f9744f260 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.217003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.217641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.217948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.218220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.218799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:49.220167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:49.229565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:49.231040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:49.231980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 381/780] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:48 2022] GET /identity/v3/services/network => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:49.238728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.259952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a13e8-c483-48b8-9acf-7a8d73021f6a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.294809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:49.312125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.313133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.313596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.322034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.328081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:49.333235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:49.335059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 400/781] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:49.337572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.406320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b11b6-73be-484e-9742-a2f91e60d6a5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.422042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:49.467575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.468577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.469059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.469534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.470669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] RBAC: Adding query filter params (name=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:49.471203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:49.474598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:49.477208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab32d83-7a74-4767-8e67-ba680bd01e3d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.487264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 382/782] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:49.487902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.530969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.533355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.533855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.534277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.549374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:49.556590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:49.557769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:49.559415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:49.562505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 401/783] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:49.563463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.620191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53938ea1-e1f3-4f01-85b0-5af2eac06b71 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.624057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:49.659519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.660882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.661584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.662411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.664017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:49.664736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:49.666550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:49.677076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 383/784] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:49.678508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.684317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5a9f5-6922-4229-b464-1300b2f06ea2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.709590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:49.722286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.722943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.723223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.723542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.724165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] RBAC: Adding query filter params (type=network) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:49.724463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:49.726328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:49.733041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 402/785] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:49.734142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.800621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cc3d8-c076-435e-886f-4506585072ed None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.846455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.847923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.848393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.848853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.850216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:49.850713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:49.853076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:49.866361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 384/786] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:49.867486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:49.883903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d62d-e485-45e3-be9a-4aa4912c8f6a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:49.905856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:49.934664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:49.935462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:49.935748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:49.936019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:49.950623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:49.952109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:49.968144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:49.969144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:49.970096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:49.970096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:49.970096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:49.970096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 403/787] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:49.970758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.009941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383ffe55-f3f5-48be-8a9f-2500e4e4d84a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.014244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.052492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.053518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.054118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.054620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.055557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] RBAC: Adding query filter params (interface=public, service_id=e65d63b90ddf4f0fad83fdaa29fc04b6, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:50.056078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.057709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.065040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 385/788] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/endpoints?service_id=e65d63b90ddf4f0fad83fdaa29fc04b6&interface=public®ion_id=RegionOne => generated 194 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.066159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.082063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2296f9fd-f405-4c4b-9c31-c77a69671bcc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.122155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.123089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.123373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.123650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.138520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.140510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:50.162379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:50.163414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:50.164395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:50.164395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:50.164395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:50.168845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 404/789] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:50.169779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.178302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.195220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e212201c-1fbf-4464-83f4-03765e1f427f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.230707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.231807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.232243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.232667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.233464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:50.233976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.235811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.250075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 386/790] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.251177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.271897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.281757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592c95c4-b8a0-4270-85f4-bce33173fe79 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.317308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.318451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.318875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.319475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.320618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.322187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.334176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 405/791] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.335405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.373079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01698fa-54bc-4183-8a3a-56a32d39ff02 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.414115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.415020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.415496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.415986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.416849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:50.417323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.419393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.438258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 387/792] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.440156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.457638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.550561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7ead5-4f01-4b60-bb5d-9f8ce47ea0a3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.559652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa7e568-9d2b-410b-bfd1-f5f94c1f849e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.604890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.606376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.606665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.606949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.619675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.622036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.631772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.632431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.632714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.632990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:50.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:50.636955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:50.637935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 388/793] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:50.642150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.643116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.643683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:50.660203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:50.662452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:50.664627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 406/794] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:50.675336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.694158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.723807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.759749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff77a5-12fa-44cb-ac00-73e4e40f8cc7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.794383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd4615-9ae7-43cc-b1df-d809ff8d706b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.803563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.804409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.804936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.805355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.806312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:50.806885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.808746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.830466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 389/795] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.836254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.858491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.859534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.859951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.860326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:50.861169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:50.861580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:50.863303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:50.899462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 407/796] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:50.906130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:50.951346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fa009-73b8-4036-a7c9-057109619f86 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:50.987574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:50.990977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:50.991975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:50.992464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:50.992966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.005879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.007759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.016882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d2c8670d-889b-44e9-906d-7361b3191504 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:51.024923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:51.025912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:51.026880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:51.026880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:51.026880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:51.029368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 390/797] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:51.035444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0810c-ac1f-4c76-a11b-63ca1c2be06a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.036060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0810c-ac1f-4c76-a11b-63ca1c2be06a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.051718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0810c-ac1f-4c76-a11b-63ca1c2be06a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.051718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 391/798] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:51.051718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0810c-ac1f-4c76-a11b-63ca1c2be06a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.052004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6f471-161c-451e-bc0a-8b7e60665cc5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.090226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.090874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.091206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.091412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.102918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.104494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.158121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0810c-ac1f-4c76-a11b-63ca1c2be06a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.199375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.200867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.201359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.201885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.202783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:51.203302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.205211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a1d79-a2f8-49d2-afd6-1e237fd09656 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.224518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 392/799] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:51.226345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d330-f751-4530-9a55-538e58192a8d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.226847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d330-f751-4530-9a55-538e58192a8d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.227329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017d330-f751-4530-9a55-538e58192a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.247174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a017d330-f751-4530-9a55-538e58192a8d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:51.247722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:51.248730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:51.249640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:51.249640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:51.249640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:51.250396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a017d330-f751-4530-9a55-538e58192a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:51.253865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 408/800] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:51.254490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.287417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.322414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 393/801] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:51.324901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c464dd1-f7fd-495d-8f95-07d4be82f8f2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.325408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c464dd1-f7fd-495d-8f95-07d4be82f8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.325946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c464dd1-f7fd-495d-8f95-07d4be82f8f2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.331419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 394/802] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:51.331419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c464dd1-f7fd-495d-8f95-07d4be82f8f2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.386725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cae5b3-4f8c-4200-96b5-3dc6292461db None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.435607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.451562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.452767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.453314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.453835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.455241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c464dd1-f7fd-495d-8f95-07d4be82f8f2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.462852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.466218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-36b8b3d1-7d41-43a0-b0fd-169be9e9cdac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:51.486086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:51.487139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:51.488109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:51.490562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 409/803] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:51.492656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.493086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.493510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.503006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.503691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.504070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.504425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.505150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:51.505527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.507052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.515965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:51.518291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:51.520243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 395/804] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:51.521227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.563231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.601883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 410/805] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:51.601883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.663835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236030e2-228b-4362-b694-74b7da1d6c9d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.702821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.703768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.704242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.704707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.705609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:51.706124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.707475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3706a6-73e9-4e7e-8ab4-9630b8da7da9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.707967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.721425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 396/806] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:51.722623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.755014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.755620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.755972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.756325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.765047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.766724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:51.780295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:51.781294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:51.782218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:51.784044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 411/807] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:51.784659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.841125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162aeef-05d5-47bf-86c8-948c99b08495 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.877933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.878866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.879313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.879761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.893151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:51.895602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:51.898798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:51.899970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:51.900906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:51.901645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212c39a1-8a86-4473-87db-3cdf6ad7815e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:51.901906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 397/808] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:51.901906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:51.933083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:51.936153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:51.940494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:51.940494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:51.940494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/6a7f47fa1a8e4902a1a68219366274c2/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:51.940494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=6a7f47fa1a8e4902a1a68219366274c2, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:51.940972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.008078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3e7414-63ce-4a5b-a3b0-47cd55af2c01 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.053767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 412/809] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:51 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/6a7f47fa1a8e4902a1a68219366274c2/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:52.053767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.081516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.081516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.081516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.081516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.081989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:52.081989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.081989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.085907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 398/810] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:52.087039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.107854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:52.126505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:52.209075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e3228-f608-4b9d-8574-1c654d825545 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.212472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f45a3f-0fe1-4bac-830f-0217e5723a85 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.254658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.255603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.256125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.256570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.257232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.258752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.259273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.260143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.260429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.260709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.261504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:52.262550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.263220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:52.272619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:52.273680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:52.274598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 413/812] 10.222.0.52 () {56 vars in 1129 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:52.275414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 399/812] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:52.275486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.282654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.382289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:52.396267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:52.401582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e88847-7ac1-409d-99c6-59e8d04d9a81 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.410535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00ee907-25f5-41b0-91f3-001a11415192 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.440121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.441108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.441108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.441407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/3002491f2c61493890d2e4272e9ba522/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.451243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.452290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.452802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.453299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.460715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=3002491f2c61493890d2e4272e9ba522, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.462236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.466420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.468167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:52.485568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:52.489932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:52.491328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:52.491328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:52.491328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:52.491328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 400/813] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:52.491328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.574539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 414/814] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:49:52 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/3002491f2c61493890d2e4272e9ba522/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:49:52.576837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.653325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c44e0e-682d-4ff4-9b14-fa0d2b273611 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.718861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:52.733983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.734618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.734903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.735184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.735949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:52.736260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.737655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.744271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 401/815] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:52.749915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.768312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e487d3da-6a8e-45fc-826f-7ab394820211 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.814335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.815294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.816238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.816719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.817723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:52.818383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.820493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7afcbf-9074-43d0-b533-1931be86a410 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.840616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 415/816] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:52.840616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c965a-9c11-420f-9dcf-f6949e41ad74 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.840616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c965a-9c11-420f-9dcf-f6949e41ad74 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.840616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c965a-9c11-420f-9dcf-f6949e41ad74 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.841213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 416/817] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:52.844325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c965a-9c11-420f-9dcf-f6949e41ad74 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:52.863928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:52.877384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db23a5-de1c-4f3d-abfd-e88d5f91cf03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:52.917203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:52.917985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.918279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.918642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.919276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:52.919583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:52.920949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d83f0-4c97-42a8-af58-a8f8745fe3bc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:52.926711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 402/818] 10.222.0.52 () {56 vars in 1124 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:52.932522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:52.932813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:52.933752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:52.956137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:52.958078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:52.990488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c965a-9c11-420f-9dcf-f6949e41ad74 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.026584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:53.035117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:53.035812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.036094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.036361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.045081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:53.046675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:53.052799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 403/819] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:53.052799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:53.059925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:53.060909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:53.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 417/820] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:49:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:53.070722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:53.181523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9924be93-9c31-4fbc-afe5-4b9575ea4f57 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.209758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:53.228596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:53.229463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.229942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.231047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.241527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:53.241527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:53.247758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f5fa71-2b46-43d7-90c9-0e01e1e61faa None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.294143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:53.300190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.300190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.300190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.300190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:53.300190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:53.300190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:53.319135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 418/821] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:49:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:53.325848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:53.430383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 404/822] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:49:53.431485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:53.449463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf432f6b-b4e3-44b4-bf5f-665682501e87 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.490409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:53.506197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:53.507396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.507933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.508425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.546487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:53.599218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:53.601578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:53.675983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d85474-5794-43e6-9c9d-5902a53b392d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.729168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 419/823] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:49:53.735091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:53.761684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:53.762362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.762647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.762925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.763519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:53.764920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:53.770765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:53.775785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:53.777317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:53.788730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 405/824] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:53 2022] GET /identity/v3/services/network => generated 88 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:53.789535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:53.908660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13b694-b2f9-4979-9b41-64774a1f036e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.944237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a0cf0e-6543-4f53-8d45-6e3ca47580e2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:53.947271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:53.947513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:53.947910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:53.948190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:53.950302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:53.965326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:53.970191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:53.998922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:54.000130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:54.001238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:54.001238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:54.001238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:54.008222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 420/825] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:54.010915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.012409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.013027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.014261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.015328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.016613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] RBAC: Adding query filter params (name=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:54.019182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.021444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.033375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 406/826] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:54.039654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.179109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f469b7e6-101b-4bc9-a8b4-93825ceea934 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.180954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.217371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.218325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.218844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.219338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.220187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:54.220724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.222467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.228400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 407/827] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/services?type=network => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:54.242148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.245020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724fab6e-ac70-42da-8a3b-4fc66578e25a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.314949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.317051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.318262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.318944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.319365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.320368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:54.321021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.324029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1204485-6f79-4a93-b3bc-55ae2b37da68 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.349445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 421/828] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:54.357653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da556187-cccb-4ea1-9d30-d2d6e85b5328 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.357653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da556187-cccb-4ea1-9d30-d2d6e85b5328 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.357653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da556187-cccb-4ea1-9d30-d2d6e85b5328 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.357653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 422/829] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:54.358385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da556187-cccb-4ea1-9d30-d2d6e85b5328 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.412558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae710931-deb5-4016-92bc-e559ec0fc3ee None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.459270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.460263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.460560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.460967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.461823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.463936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.480834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da556187-cccb-4ea1-9d30-d2d6e85b5328 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.517934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.518659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.519011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.519357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.521539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.527644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.529108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e18bd7f4-cb63-4275-b7f9-114c94c2b5b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:54.540298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:54.541299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 423/830] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243ee1a-44a8-4f63-8b6d-6052209ea8b5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.547297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243ee1a-44a8-4f63-8b6d-6052209ea8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.547937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243ee1a-44a8-4f63-8b6d-6052209ea8b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.558975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 408/831] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:49:54 2022] POST /identity/v3/endpoints => generated 337 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:49:54.561244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.565505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8243ee1a-44a8-4f63-8b6d-6052209ea8b5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:54.566951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8243ee1a-44a8-4f63-8b6d-6052209ea8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:54.692918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:54.716542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46697226-40d0-4b12-bdec-201222bdfc95 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.731758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 424/832] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:54.783920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.784538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c41295-63e5-4f12-9021-c511d8f576b5 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.785002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.785094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c41295-63e5-4f12-9021-c511d8f576b5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.785167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c41295-63e5-4f12-9021-c511d8f576b5 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.785235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.785759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.786200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:54.786508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:54.786847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 425/833] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:49:54.787952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:54.795201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.795806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:54.796290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:54.809492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 409/834] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:54.813702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:54.830620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:54.832140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:54.935091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 426/835] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:54.957066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65baa0-e4bf-49fc-8a6f-f6f30053ce39 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:54.998234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:54.999260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:54.999819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.000307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.001283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=e1a623f1fc434449a76106e74756e35c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:55.001825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:55.003609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:55.010531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 410/836] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:54 2022] GET /identity/v3/role_assignments?user.id=e1a623f1fc434449a76106e74756e35c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:55.061025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.121047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.177442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6309f11-e44d-4f46-a243-08939da3fbcb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.213868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.216233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.216879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.217164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.217670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.227625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.233192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:55.238082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:55.239031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:55.239890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 427/837] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:55.244996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.350525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c842da7-8b18-4b0f-90d9-65e061a49dfb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.376370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef45b9-5f81-426a-9e8f-d9be63b0122e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.389183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.399723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.400846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.406968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.406968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.416598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:49:55.417146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.418110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.418595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.419141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.420007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:55.420511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:49:55.422099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:55.423056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:49:55.423905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:55.425138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:55.428311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 411/838] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:55.429213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.441861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 428/839] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:55.443988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.525063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.559727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4966a-6e37-4b93-aded-42da01000c09 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.578949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc604602-12d9-4721-a8c3-b942e575f1a4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.599213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.600245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.600764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.601277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.614106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:55.615967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:55.623511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.623885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.624173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.624450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.625116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:55.625419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:55.626902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:55.635991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:55.637716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:55.638739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:55.638739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:55.638739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:55.645124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 412/840] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:55.645124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.645770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 429/841] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:55.646929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.676798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.768268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa83062b-c656-4373-a6e9-d29f0d907f9e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.768499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38bbc0-a9ee-469d-bc6e-487f78d2cc8e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.799832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.806384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.807266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.807769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.808846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.814820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:55.816586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:55.816586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:55.816586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:55.816586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:55.816900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:55.818452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:55.822916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:55.824876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:55.835619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 430/842] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:55.840629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:55.841759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:55.842864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:55.842864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:55.842864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:55.843403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.849511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 413/843] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:55.851635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:55.942930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:55.959983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48220a81-46cf-4d23-ba32-55f65cb83075 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.977995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90067e9d-c32d-46e9-8603-c3ed35d69e7d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:55.999687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.000769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.001054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.001630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.011235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.012693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.014515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.015120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.015730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.015730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.016299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:56.016607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.017961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-1f6a02e3-9fb1-4b0a-855b-0cf23bd7f5f7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:56.025488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:56.027247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 00:49:56.028275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:56.032317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 431/844] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:56.033714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a42d-53b0-410b-9eb6-15378f57535e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.034025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a42d-53b0-410b-9eb6-15378f57535e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.034300 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856a42d-53b0-410b-9eb6-15378f57535e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.039575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 414/845] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:56.039710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 432/846] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:56.039809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.040248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a42d-53b0-410b-9eb6-15378f57535e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.091938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.155032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856a42d-53b0-410b-9eb6-15378f57535e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.156386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c2cd8-3f6e-4ade-98ad-e437e74bed7d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.192398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.193152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.193382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.193662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.194198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.194830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.195110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.195382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.196143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:56.196453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.197967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.204288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.206131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-560e7890-9ca2-4218-952b-430421ec2e12 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:49:56.219264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:56.220482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:49:56.221647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:56.223819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 433/847] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:56.224453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 415/848] 10.222.0.52 () {56 vars in 1135 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:56.225366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.225560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.226038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.226313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.234089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.244788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:56.246547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:56.320719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 434/849] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:56 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:56.321911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.341583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.344976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e9e66-fd20-4eec-bfac-534e6b8789e9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.388020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.388997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.389346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.389717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.444901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a0788d-d7fe-4f0d-84c8-80680b37030f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.475105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.477004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.490849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.491906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.492192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.492470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.493488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:56.493834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.495592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.499516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.525896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 435/850] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:56.529051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.597579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.602058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 416/851] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:56.607771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.660424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57d4c4-23ca-4cb4-a964-3776e890906d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.709291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.710183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.710637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.711086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.748955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fedd1fc-f567-474b-a13d-d8252c5b2d00 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:56.804479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:56.804930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.805217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:56.805502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:56.805876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:56.819423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.821962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.822987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:56.825642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:56.847095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:56.848113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:56.849016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:56.849016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:56.849016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:56.849016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 417/852] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:49:56.849255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.909194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:56.944080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 436/853] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:49:56.944922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:56.970255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42ee61-9744-4b28-9d20-7b2135f800b6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.018471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.019550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.020121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.020540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.021429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.023788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:57.037693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:57.038765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:57.039614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 418/854] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:57.040421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.055341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.066126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55c38b-ff54-4da9-afa9-a1463fd471c0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.102285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.102896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.103178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.103540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.104097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.104400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.106526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.117834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 437/855] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.118652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.157446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611f4b9-71d6-4b06-ac72-17bfeb7985ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.174661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.192841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.193480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.193785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.194069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.204262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.205720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.219626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c831f34-99e8-4fd6-85af-282bcfd15aaf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-632da7b2-8546-4298-96e1-8abaa6099df8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:49:57.221398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:57.222389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:49:57.223281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:49:57.223281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:49:57.223281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:57.223281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 419/856] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:57.223899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0da80-f9c7-43a7-840c-6ffa09b77b42 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.224186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0da80-f9c7-43a7-840c-6ffa09b77b42 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.224463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0da80-f9c7-43a7-840c-6ffa09b77b42 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.225891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 420/857] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:57.230990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0da80-f9c7-43a7-840c-6ffa09b77b42 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.254787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.255398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.255679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.255953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.256583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.256891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.258309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.267786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 438/858] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.267786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.283522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.334687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0da80-f9c7-43a7-840c-6ffa09b77b42 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.367512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baaa4e0-acba-451d-8307-a9be4026adac None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.372270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.372926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.373209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.373486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.380701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.382201 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c243e9a6-4877-4717-992d-1221e4d0b378 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:57.393936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:57.395334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 421/859] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:57.396207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.396764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.396764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.411402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.412217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.412502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.412784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.413430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.413964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.415340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.415721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:57.416943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:57.428556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 439/860] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.429386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.445535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.490276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 422/861] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:49:57.491046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.512440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.536258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d0dd6-d761-47df-957d-dfffca6cc6cc None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.572972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.574074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.574652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.574993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.575627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.575989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.577366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.582653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 440/862] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.583664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.615531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581851ac-d2fb-4cb1-8950-451f42c9e2f8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.661320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.662361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.662801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.663263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.664254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.664716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.666684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.681590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 423/863] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.688407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.712651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.752009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae892e2-7be1-4c55-8234-e842584d522b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.798622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.799285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.799946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.800221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.809576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.811639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.815062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:57.825607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d66984-bfd7-420b-b204-7ebe88e040df None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:49:57.829291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:57.830671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:49:57.831777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:57.833783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 441/864] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:57.834684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.871372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:57.872310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:57.872700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:57.873207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:57.874092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=6a7f47fa1a8e4902a1a68219366274c2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:57.874515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:57.876188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:57.883429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 424/865] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/role_assignments?user.id=6a7f47fa1a8e4902a1a68219366274c2&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:57.884435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:57.970912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b852c5-54f9-407d-b162-e1cfd6ea6ea9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:58.014278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:58.015003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:58.015418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:58.015814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:58.016696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:58.018359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:49:58.029548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:49:58.030787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:49:58.031889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:49:58.033952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 442/866] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:49:58.039603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:58.115918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:58.143972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a86f9b-7679-48dc-aff7-df9b604523a0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:58.148220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:58.193090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa639a4-56cf-4598-9809-adacdc65a2f2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:58.217157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:58.217816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:58.218150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:58.218505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:58.219657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:58.219657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:58.221037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:58.248267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 425/867] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:49:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:58.248482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:58.248482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:58.248482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:58.248482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:58.248758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:58.248758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:58.250374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:58.251928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:58.268329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 443/868] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:58.273544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:49:58.383197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67366ec0-4d7d-44cf-8313-6efa7e95001d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:58.404699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0743fe-4926-4b94-abf3-a594f991b8bb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:49:58.422448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:58.423232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:58.423515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:58.423791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:58.424512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=3002491f2c61493890d2e4272e9ba522) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:58.424820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:58.426424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7705-d4dd-4186-9e3f-34c0348accf5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:58.434716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 426/869] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:49:58 2022] GET /identity/v3/role_assignments?user.id=3002491f2c61493890d2e4272e9ba522&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:49:58.447260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:49:58.457619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:49:58.458283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:58.458560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:58.461157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:58.461157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:49:58.461157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:49:58.461408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf4410-1e1b-4abf-ad4e-72370404c095 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:49:58.469933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 444/870] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:49:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:49:59.619802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09774-a8ae-48fd-a50b-790fc1e9cc2f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:59.620476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09774-a8ae-48fd-a50b-790fc1e9cc2f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:59.620476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09774-a8ae-48fd-a50b-790fc1e9cc2f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:59.622753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 427/871] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:49:59.628807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72b707-69a2-4a17-8980-7ce78b43cde8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:49:59.629323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72b707-69a2-4a17-8980-7ce78b43cde8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:49:59.629323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f72b707-69a2-4a17-8980-7ce78b43cde8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:49:59.648349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3f72b707-69a2-4a17-8980-7ce78b43cde8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:49:59.649530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3f72b707-69a2-4a17-8980-7ce78b43cde8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:49:59.722953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 445/872] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:49:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:00.001684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09774-a8ae-48fd-a50b-790fc1e9cc2f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.011785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.036803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f565ec-666e-4c3f-bb76-12a969944894 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.037408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f565ec-666e-4c3f-bb76-12a969944894 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.038203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f565ec-666e-4c3f-bb76-12a969944894 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.040182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 446/873] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:00.045904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.046264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.046646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.069299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:00.071356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:00.120734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09774-a8ae-48fd-a50b-790fc1e9cc2f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.157910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 447/874] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:00.166050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5977b-8f63-4bde-a339-b9828932c757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.166951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5977b-8f63-4bde-a339-b9828932c757 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.168121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5977b-8f63-4bde-a339-b9828932c757 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.168799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5977b-8f63-4bde-a339-b9828932c757 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.170093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5977b-8f63-4bde-a339-b9828932c757 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.172355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5977b-8f63-4bde-a339-b9828932c757 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.245161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 428/875] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:00 2022] POST /identity/v3/services => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:00.434651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.446282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3005-630a-4c7f-ae58-ff60b06eeffe None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.446571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3005-630a-4c7f-ae58-ff60b06eeffe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.446839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b3005-630a-4c7f-ae58-ff60b06eeffe None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.448575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 429/876] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:00.454098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b7c6f-625e-416e-8988-6104b41f54f1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.454463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b7c6f-625e-416e-8988-6104b41f54f1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.454818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b7c6f-625e-416e-8988-6104b41f54f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.481339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a1b7c6f-625e-416e-8988-6104b41f54f1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:00.485766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a1b7c6f-625e-416e-8988-6104b41f54f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:00.496539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.550440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d281d6-14ec-4447-9cac-39b68f80ddb2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.555471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.573780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 430/877] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:00.573780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da62f-7197-494b-b6ff-db20867727db None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.573780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da62f-7197-494b-b6ff-db20867727db None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.573780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27da62f-7197-494b-b6ff-db20867727db None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.578357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 431/878] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:00.579620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.579904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.580193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.596198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.597165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.597665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.598142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.598990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.599680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:00.600887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:00.601077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:00.613980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:00.615056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:00.615902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:00.618978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 448/879] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:00.625116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.689715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 432/880] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:00 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:00.749794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1dbb4-9868-4164-b2f9-4fae28cadd01 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.770252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.790162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.790779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.791063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.791343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.792015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:00.792614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.794273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.799525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 449/881] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:00.805086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.878064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:00.919518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d68f730-af23-4172-90d2-93bcf9bbd667 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:00.948372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:00.965111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:00.966426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:00.967171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:00.967696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:00.968968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:00.969777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:00.971406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:00.977609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 433/882] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:01.001319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:01.033859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af79ac6f-be95-4d13-9b79-4cfab42d6cea None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:01.086874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:01.087580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:01.087866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:01.088146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:01.089164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:01.090864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:01.121080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdc3418-ed2a-47b6-a287-37a2ef1a875f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:01.159980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:01.160623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:01.160911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:01.161192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:01.173871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:01.182834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:01.183946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:01.184806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 434/883] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 30 00:50:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:01.186741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:01.216426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 450/884] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:00 2022] POST /identity/v3/services => generated 269 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:01.254650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:01.318785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2ed3a-efdd-4b5e-8737-48784ca07309 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:01.365907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:01.366531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:01.366810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:01.367085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:01.367742 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:01.368047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:01.369405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:01.379568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 435/885] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:01.385292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:01.439537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:01.502291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd7d90-6dbf-4123-8709-e80e4c7b2863 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:01.540289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:01.540937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:01.541221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:01.541496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:01.553713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:01.555200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:01.570334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:01.571311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:01.572302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:01.572302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:01.572302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:01.572302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 451/886] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:01.578603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:01.688719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300d953-7b54-4d8e-ac3f-b6fc5b5ebc61 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:01.691186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:01.725200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:01.725891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:01.726182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:01.726460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:01.727107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:01.727414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:01.728898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:01.742374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 436/887] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:01.749066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:01.850216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:01.860158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62353104-0a1f-4347-84c9-e6be84bbe424 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:01.896483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:01.897127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:01.897408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:01.897681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:01.906977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:01.908433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:01.920676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:01.921650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:01.922621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 452/888] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:50:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:01.927825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.036061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72f4051-710c-4c49-8e25-9d61208633c1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.071506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.072131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.072414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.072692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.073521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:02.074267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.075710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a86f0-6966-4a37-8c98-f53ebe181c97 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.094881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 437/889] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 30 00:50:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:02.100553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.212237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003103d9-aadb-4ffb-a4ab-166022ef5e15 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.249316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.249960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.250243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.250516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.268577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee40368-1ef3-4ae2-84e1-e267f523d6b3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.268902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee40368-1ef3-4ae2-84e1-e267f523d6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.269148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee40368-1ef3-4ae2-84e1-e267f523d6b3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.271334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 438/890] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:02.278395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.278805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.279195 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.285684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.299493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:02.300351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:02.327851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.339931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.341925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.405903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 439/891] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:02 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:02.458018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 453/892] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:02.464058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.581652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfc30e-c514-4966-9ded-466d4ce3ba2b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.619034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.619678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.619963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.620855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.631954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.633411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:02.648994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:02.649991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:02.650905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:02.650905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:02.650905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:02.650905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 440/893] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:02.656832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.711643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.739924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.744023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.769650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da6573-b31b-450d-a5c3-7e153080e6bf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.806174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.806789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.807067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.807399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.808713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:02.808897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.810372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.822949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 454/894] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:02.830294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.860690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df2477-7862-4c88-b0f0-5bd602c80da5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.901660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.902863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.903372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.903872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.904669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.906293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:02.915957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:02.917065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:50:02.918766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:02.926277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 441/895] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:02.927450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:02.941177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e86b5-3c71-4960-83de-757962b39764 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:02.954870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:02.978510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:02.979156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:02.979530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:02.980364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:02.987823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:02.989299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:03.002657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:03.003597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:03.004522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:03.008745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 455/896] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:03.009751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.015546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:03.053754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc0032-8343-4cd8-bb5f-1e666e422651 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.094991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.095974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.096262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.096544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.097490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:03.098003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.099401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186d968-3db7-4a9a-ad70-d7d047071100 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.104516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 442/897] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:50:02 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:03.106200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5bc0e-4109-41fa-9644-b1c0bc5e8cb9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.106492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5bc0e-4109-41fa-9644-b1c0bc5e8cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.106773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e5bc0e-4109-41fa-9644-b1c0bc5e8cb9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.108132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 443/898] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:03.113404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.113631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.113932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.127092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39a046-71e5-4e11-a162-f1147520936b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.133663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:03.134911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:03.165782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.166672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.166990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.167331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.168033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:03.168394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.169874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.188463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 456/899] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:50:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:03.189155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.217871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 444/900] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:03.219725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.226299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:03.300590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717a0823-eb6c-4814-8935-6e6d2c6993d3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.307378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:03.340609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.341224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.341504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.341800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.342882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:03.343190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.345101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b8bfe-e5bd-438e-aa9b-9f54a7a01357 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.350692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 457/901] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:50:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:03.351869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a66de-c2a5-4895-8bcd-eb2219e99ead None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.352154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a66de-c2a5-4895-8bcd-eb2219e99ead None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.352427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62a66de-c2a5-4895-8bcd-eb2219e99ead None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.353787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 458/902] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:03.358540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a66de-c2a5-4895-8bcd-eb2219e99ead None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.360428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f23fac5-50ab-43bc-9d06-317b828af8ad None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.399957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.400513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.400780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.401062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.402349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] RBAC: Adding query filter params (role.id=c578d26674f3470abc3ec8613c828c95, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=3002491f2c61493890d2e4272e9ba522) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:03.402671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.404101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c098faa-fc5d-467b-8b74-b61797c57a2a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.417769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 445/903] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:03 2022] GET /identity/v3/role_assignments?user.id=3002491f2c61493890d2e4272e9ba522&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=c578d26674f3470abc3ec8613c828c95 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:03.417769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.417769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.417769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.442983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:03.444200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:03.501375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:03.508940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62a66de-c2a5-4895-8bcd-eb2219e99ead None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.537397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 446/904] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:03.538124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.547416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.548042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.548476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.548747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.549462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] RBAC: Adding query filter params (interface=public, service_id=e94f8f075f264025bdc66135a63cff04, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:03.549794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.551209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.563649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 459/905] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:50:03 2022] GET /identity/v3/endpoints?service_id=e94f8f075f264025bdc66135a63cff04&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:03.569176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.620820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:03.675578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79ea68-f151-4ca4-a107-bda8951c6a49 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.693434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479dad9-fd2d-4f4f-b82b-ee2224e9b511 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.720025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.721163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.721593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.722317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.723580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.725955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.730416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.731080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.731367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.731644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.732259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.733643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:03.741765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 460/906] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:50:03 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:50:03.805220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 447/907] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:03 2022] POST /identity/v3/services => generated 241 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:03.821897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:03.856052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:03.947728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1ccdc-4dc3-4bf8-9dac-66bc3a9342b4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:03.990395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:03.991037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:03.991317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:03.991588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:03.992157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:03.993529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:04.006983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:04.008199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:04.009892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 461/908] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Aug 30 00:50:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:04.015063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:04.130418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a553dcf-744d-4f6a-8aab-1e84bc42b73b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:04.148499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:04.168485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:04.169390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:04.169911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:04.170367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:04.171169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:04.171664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:04.173198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:04.178882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 448/909] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:04 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:04.185415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:04.281185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:04.297661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd041e99-8576-4a7f-b68d-c8c8dadee4d7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:04.333473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:04.334134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:04.334696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:04.334985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:04.335630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:04.335939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:04.337331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:04.344298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 462/910] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:04.350295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:04.459749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6207c-6c94-45fb-993e-fbf45a20fd22 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:04.497159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:04.497800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:04.498087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:04.498363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:04.499076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] RBAC: Adding query filter params (interface=public, service_id=806e885bc3af48519ccc394ab8f565b2, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:04.499388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:04.500751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ff4c6e-b0f9-43fd-aee7-3656b3667a2b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:04.506308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 449/911] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:50:04 2022] GET /identity/v3/endpoints?service_id=806e885bc3af48519ccc394ab8f565b2&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:04.512000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:04.623878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8681f232-5586-46cd-a955-568302d156f5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:04.660329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb49d4-f476-4162-bc0b-1137be03baa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:04.661318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb49d4-f476-4162-bc0b-1137be03baa6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:04.661837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb49d4-f476-4162-bc0b-1137be03baa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:04.662318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb49d4-f476-4162-bc0b-1137be03baa6 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:04.663132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb49d4-f476-4162-bc0b-1137be03baa6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:04.664766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb49d4-f476-4162-bc0b-1137be03baa6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:04.672545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 463/912] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:50:04 2022] GET /identity/v3/services => generated 1497 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:50:05.427486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf4c20-313b-437b-bfe4-43170833741c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.428005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf4c20-313b-437b-bfe4-43170833741c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.428112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf4c20-313b-437b-bfe4-43170833741c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.429658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 450/913] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:05.435062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d00d-96e6-40d7-af7b-3e11c0f5cd9a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.435495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d00d-96e6-40d7-af7b-3e11c0f5cd9a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.435679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e070d00d-96e6-40d7-af7b-3e11c0f5cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.455012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e070d00d-96e6-40d7-af7b-3e11c0f5cd9a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:05.456226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e070d00d-96e6-40d7-af7b-3e11c0f5cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:05.464748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:05.531931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 464/914] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:05.781035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cb0a4-9af1-4a31-a29f-2b09b94af380 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.781670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cb0a4-9af1-4a31-a29f-2b09b94af380 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.782206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576cb0a4-9af1-4a31-a29f-2b09b94af380 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.783981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 451/915] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:05.789476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73ee26-2048-4735-8bd0-e984bc870eb6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.790043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73ee26-2048-4735-8bd0-e984bc870eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.790550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73ee26-2048-4735-8bd0-e984bc870eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.802584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cb0a4-9af1-4a31-a29f-2b09b94af380 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:05.811122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7d73ee26-2048-4735-8bd0-e984bc870eb6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:05.812366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7d73ee26-2048-4735-8bd0-e984bc870eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:05.870040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:05.892060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 465/916] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:05.934315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576cb0a4-9af1-4a31-a29f-2b09b94af380 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:05.935784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a518e-14a6-4724-997a-60bb8f0e00c3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.936079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a518e-14a6-4724-997a-60bb8f0e00c3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.936360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a518e-14a6-4724-997a-60bb8f0e00c3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.941578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 466/917] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:05.943164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.943450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.943731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.962538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:05.964683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:05.975344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:05.975991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:05.976669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:05.976971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:05.993128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:06.000515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:06.001457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:06.002355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:06.005945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 452/918] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 30 00:50:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:06.007634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.048607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 467/919] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:06.128021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1712c9-4e0c-49ad-9e3f-b6d12bea8dd6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.130829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.165721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.167342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.167904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.168387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.169255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.169824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.171402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.178425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.181215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.183308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 453/920] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:06.188842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.294490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002dde2-379a-4eb8-992b-3d59f748e0e5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.304383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34422b14-a9f6-48cd-9f2f-2118e215d9a5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.331966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.332610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.333090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.333284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.334062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.335371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.342377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.343018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.343300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.343579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:06.345152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:06.346146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:06.346994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 468/921] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:06.348517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.363351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.365982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:06.383967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:06.389693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:06.390641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:06.390641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:06.390641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:06.390641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 454/922] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:06.390641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.412549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.422040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.457674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d42cb1-92c9-4a72-a27b-a464c4b4ce82 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.500412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.501137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.501327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.501603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.502285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.503716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.512027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360be87a-2333-4ce3-8d1e-bf769b01e1fb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:06.513642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:06.514661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:06.515502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 469/923] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:06.516310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.554961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.555542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.555789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.556067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.556705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.557020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.558564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.563709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 455/924] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:06.564525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.630748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5090440-11ff-48e4-9a5f-bf48984638c9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.652331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.662656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.669095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.669710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.670023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.670375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.671045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.671358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.672729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.673070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6005766-81d7-40d5-a6bd-d584341128e3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.684919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 470/925] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:06.685756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.709425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.710087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.710372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.710654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.711289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.711607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.712966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992257ba-15e5-43f7-8ffa-fec328cf0178 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.718532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 456/926] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:06.719810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f687be8-c7c9-4e17-9455-83e65871132f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.720110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f687be8-c7c9-4e17-9455-83e65871132f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.720393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f687be8-c7c9-4e17-9455-83e65871132f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.721775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 457/927] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:06.722472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f687be8-c7c9-4e17-9455-83e65871132f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.796064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe64c04-f158-444d-ba53-fc822e6809ca None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.831704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.832243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.832519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.832791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.833410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.833712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.835213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.840580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f687be8-c7c9-4e17-9455-83e65871132f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.841029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 471/928] 10.222.0.52 () {56 vars in 1114 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:06.841886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.875406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.876042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.876323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.876599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.883856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.885298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.897791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8d739675-a3ad-4b4d-a341-b54fe8feb843 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:06.898418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:06.899339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:06.900292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 458/929] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:06.900893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.900960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.901471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.914027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:06.920266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:06.921579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:06.948991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761ce75-3445-48b8-8c3c-5a6c89434d04 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:06.984100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:06.984715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:06.985007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:06.985281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:06.985925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:06.986229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:06.987602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:06.990250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 459/930] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:06 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:06.990910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:06.992619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 472/931] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:06.993404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.107287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd718e-cafe-44aa-a4b6-7a82473a9891 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.108027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c1a3d-b782-4d69-819b-b908b3e5bfa1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.125911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:07.141839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.142453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.142736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.143056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.143674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.144235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.144846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.145033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.145128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.145393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.146174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:07.146488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.147955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.150652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:07.173299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 473/932] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 30 00:50:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:07.173299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.222891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 460/933] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:50:06 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:07.223718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.303878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abeaf1d-597a-4074-ae4a-4b62c5b98b8c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.350431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.351061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.351266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f6dea5-a2ef-4c14-ad8c-ca9fb6fa18b7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.351371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.351602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.352314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] RBAC: Adding query filter params (interface=public, service_id=824450beaea94868a523052eddceb74d, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:07.352626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.354162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.360210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 474/934] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/endpoints?service_id=824450beaea94868a523052eddceb74d&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:07.360991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.402699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.404093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.404093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.404093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.409591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:07.419740 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.423249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:07.436738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:07.437855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:07.438769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:07.438769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:07.438769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:07.438769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 461/935] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:07.438769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.496045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:07.499180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6444bbcc-b1f2-4841-a60d-e10881efd4e6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.539598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.540405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.540858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.541151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.542387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.543682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:07.553538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:07.554756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:07.555678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 475/936] 10.222.0.52 () {56 vars in 1132 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:07.556489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.568797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a53785-3a87-4927-9dbd-30f00c08ac58 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.610879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.611747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.612192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.612776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.613548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.615107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.622337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 462/937] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:50:07.623148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.691042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:07.695882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f79b031-dfa2-416e-9bd8-6f051f2254f1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.749537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.750519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.750952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.751376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.752905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:07.753393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.755670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.767229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1fd0c6-2362-4f45-b10d-5a7f02f7afba None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.773408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 476/938] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:07.778365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:07.779957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.808811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.809932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.810413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.810888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.811804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:07.812290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.814142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.820699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 463/939] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:07.826295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:07.922852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b35a38-3dc0-4759-b7de-9bc046804ab7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.936534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff110f04-6a78-43fc-b841-45a79353a05f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:07.968739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.969406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.969807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.970090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.977797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.979339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.986775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:07.987449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:07.987746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:07.988032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:07.988668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:07.988978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:07.990370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:07.992064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:07.992977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:07.994054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 477/940] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:07.996967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 464/941] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:08.002116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:08.003183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:08.032083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:08.043979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:08.111940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df5a7d-4806-4790-b5c2-e9ab866b03ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:08.145989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d324de3e-866f-480b-aff0-5e26edf76383 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:08.158045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833bc4f-6d7b-45ac-91e7-80bff99fece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:08.159327 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833bc4f-6d7b-45ac-91e7-80bff99fece0 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.159616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833bc4f-6d7b-45ac-91e7-80bff99fece0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.160166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833bc4f-6d7b-45ac-91e7-80bff99fece0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.160891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833bc4f-6d7b-45ac-91e7-80bff99fece0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:08.163067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833bc4f-6d7b-45ac-91e7-80bff99fece0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:08.191714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:08.191714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.191714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.191714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.192138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:08.192138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:08.192138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:08.205805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 478/942] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:50:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:08.208573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:08.295986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:08.309788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 465/943] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:50:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:08.338998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd2a946-f7fd-4c98-9bd9-1524fe710d67 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:08.390268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311c088-5ade-4146-a5cf-e801dc16f45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:08.391688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311c088-5ade-4146-a5cf-e801dc16f45d None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:08.392305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311c088-5ade-4146-a5cf-e801dc16f45d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:08.392587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311c088-5ade-4146-a5cf-e801dc16f45d None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/3002491f2c61493890d2e4272e9ba522/roles/c578d26674f3470abc3ec8613c828c95` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:08.395365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311c088-5ade-4146-a5cf-e801dc16f45d None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=3002491f2c61493890d2e4272e9ba522, role_id=c578d26674f3470abc3ec8613c828c95)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:08.396977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7311c088-5ade-4146-a5cf-e801dc16f45d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:08.572147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 479/944] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:08 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/3002491f2c61493890d2e4272e9ba522/roles/c578d26674f3470abc3ec8613c828c95 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:50:09.255730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707961-699e-4518-8599-0f5f0f2b868d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.256248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707961-699e-4518-8599-0f5f0f2b868d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.256349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707961-699e-4518-8599-0f5f0f2b868d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.257994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 466/945] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:09.264426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9bff6-6fab-413d-a4cb-87382261de0d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.264722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9bff6-6fab-413d-a4cb-87382261de0d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.265008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9bff6-6fab-413d-a4cb-87382261de0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.284294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d8c9bff6-6fab-413d-a4cb-87382261de0d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:09.285496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d8c9bff6-6fab-413d-a4cb-87382261de0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:09.360779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 480/946] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:09.638669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707961-699e-4518-8599-0f5f0f2b868d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:09.676867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b022e86-30d9-47b9-a32d-0794632101a3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.677375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b022e86-30d9-47b9-a32d-0794632101a3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.677599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b022e86-30d9-47b9-a32d-0794632101a3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.679405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 481/947] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:09.687753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.687753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.687753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.704255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:09.705956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:09.755080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:09.765984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707961-699e-4518-8599-0f5f0f2b868d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:09.802493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:09.829776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 482/948] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:09.830816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:09.831677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:09.832169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:09.832589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:09.833961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:09.835574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:09.840572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING keystone.server.flask.common [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 00:50:09.891044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:50:09.891044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 00:50:09.891044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [parameters: {'id': '3bd0805cfb3c43d19fe6fb062686a6d8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 00:50:09.891044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3bd0805cfb3c43d19fe6fb062686a6d8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3bd0805cfb3c43d19fe6fb062686a6d8. Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:50:09.909280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 00:50:09.910335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Aug 30 00:50:09.912103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:50:09.913298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:50:09.914689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:50:09.917759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'id': '3bd0805cfb3c43d19fe6fb062686a6d8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:09.921610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:09.926590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 3bd0805cfb3c43d19fe6fb062686a6d8. Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:09.927725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 467/949] 10.222.0.52 () {60 vars in 1140 bytes} [Tue Aug 30 00:50:09 2022] POST /identity/v3/projects => generated 262 bytes in 286 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:09.933584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.072249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed870b7e-008e-48d7-a713-d457ed9dd720 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:10.113025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:10.114838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.115310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.115596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.126631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:10.128202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:10.142824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:10.143980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:10.145381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 483/950] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:10.154283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.162147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.249211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:10.309595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217f2fa8-afe8-4264-9788-08130e934dbc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:10.326321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8e2fa-8ff0-4069-a67f-5db29a87e951 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:10.358752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:10.359887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.360582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.364642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.364642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:10.364642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:10.366060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ac11b-97c6-45c3-9391-ceaba5da7148 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:10.366706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:10.367496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.368137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.368598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.369312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:10.372276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:10.382295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:10.384135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:10.386059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:10.387701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 484/951] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:10.392192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.404483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 468/952] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:10.462499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bec3a2-283b-43e4-8813-aa5775383e24 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.463128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bec3a2-283b-43e4-8813-aa5775383e24 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.464047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bec3a2-283b-43e4-8813-aa5775383e24 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.467979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 469/953] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:10.472315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a284a5-4351-4062-8631-323af92febf7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.472852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a284a5-4351-4062-8631-323af92febf7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.473264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a284a5-4351-4062-8631-323af92febf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.506762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-50a284a5-4351-4062-8631-323af92febf7 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:10.509106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-50a284a5-4351-4062-8631-323af92febf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:10.512148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:10.577282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a9306-9d57-44f0-8140-d5d762891c21 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:10.612314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 470/954] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:10.625335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:10.646210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:10.647306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.647903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.648447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.649636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:10.650264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:10.652442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee478ea-2580-4028-8daa-17cb4aa76af4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:10.659465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 485/955] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:10.665102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a284a5-4351-4062-8631-323af92febf7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.732641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d52a9-bf0d-4e6b-a753-9ea5d72803e1 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.733169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d52a9-bf0d-4e6b-a753-9ea5d72803e1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.733356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d52a9-bf0d-4e6b-a753-9ea5d72803e1 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.735400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 486/956] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:10.741764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.742388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.742874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.767797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:10.769376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:10.812689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a284a5-4351-4062-8631-323af92febf7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:10.858605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:10.859907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:10.860399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:10.860988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:10.862063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:10.862921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:10.865540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:10.876870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 487/957] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:10.877345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 471/958] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:10.883311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.924183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:10.933701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.002846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38919e-4c17-4b06-a15b-2d6ba4a8d3fb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.019112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.039420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13b7bf-7cd1-4e37-9add-788eea68fb35 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.043680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.044492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.044925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.045351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.046574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.048274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.076187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.076830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.077114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.085512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.085512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.085512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:11.089761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:11.090802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:11.094802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 472/959] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:11.096988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.166022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 488/960] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:50:10 2022] POST /identity/v3/endpoints => generated 349 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:11.169527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.186982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.240968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4418ee3e-b987-4fe9-abcc-87914c14b53a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.303354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.304251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.304550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.304836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.305941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:11.306260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.308338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.316196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 473/961] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:11.321189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.327504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80204ff1-4e21-4269-8884-ee525f3ed682 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.376482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.377511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.377879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.378149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.394125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:11.402104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:11.403093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:11.403950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:11.406963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 489/962] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:11.412537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.447485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.468338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423fb699-3b5b-46dd-9fdf-113a3d483b2c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.504916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.515676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.516649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.516772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.517179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.518023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:11.518668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.519950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.525111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 474/963] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:11.556911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e437dc3-023d-48b4-bd74-8820706e75b4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.605087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.605921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.606384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.606773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.607705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:11.608124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.609685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.620562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 490/964] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:11.626341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.744418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aecab30-6932-454d-8b15-74ea6459e04d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.783156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.783809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.784138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.784421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.796539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.798122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:11.814818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:11.816020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:11.817111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:11.817111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:11.817111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:11.817111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 475/965] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:11.821699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:11.926087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:11.936290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a304cb8-68d2-4a1d-b8d7-3bfeae633eaf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:11.973247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:11.973905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:11.974198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:11.974480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:11.975128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:11.975433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:11.976808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:11.989715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 491/966] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:11.996199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.045759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:12.107470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781ffb2-f5e2-4b4c-8b55-df494bf84b10 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.144136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.144839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.145131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.145413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.154121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.155589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:12.168377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:12.169641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:12.170883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 476/967] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:50:11 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:12.175536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.287577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3cca6-c40e-4446-87d2-1d45d01853c9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.324994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.325618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.325953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.326230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.326964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:12.327272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.328802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390a5d4-bacd-44ed-837f-73d0eea698ef None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.349156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 492/968] 10.222.0.52 () {56 vars in 1149 bytes} [Tue Aug 30 00:50:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:12.355958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.466643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f6c73-e8e9-41b6-947f-4f382dab7454 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.474231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:12.507677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.508302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.508591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.508874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.579907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d956e5-ffb4-4a58-801f-150b7fa89861 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.580266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d956e5-ffb4-4a58-801f-150b7fa89861 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.580473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d956e5-ffb4-4a58-801f-150b7fa89861 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.581991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 493/969] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:12.586750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.587406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.588147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.589016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.589398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.613481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:12.615036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:12.619195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:12.705888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 494/970] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:12 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:12.718774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 477/971] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:12 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:12.724397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.838410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6aa8c-3a61-4707-817d-26522af81702 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:12.875101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:12.875748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:12.876035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:12.876316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:12.885118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:12.890443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:12.891868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:12.909088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:12.918188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:12.919262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:12.919262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:12.919262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:12.919262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 495/972] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:12.922885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:12.925108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.038880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2453c-4fc4-4aa8-a0b6-d6729a715322 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.046442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.090487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.091323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.091608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.091888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.092541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:13.092848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.095419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.112361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 478/973] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:13.118624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.191624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d1a597-18c5-40d3-978e-af1048faddb5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.197039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.231264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.232128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.232591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.233351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.234046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.235505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.238151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING keystone.server.flask.common [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 00:50:13.241409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ee4ad4-56d9-488f-83be-9a63e23be1e5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.277321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.sql.core [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:50:13.277321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 00:50:13.277321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [parameters: {'user_id': '7cee85cd8e08434994972ae3aecd99fc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 00:50:13.277321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81621) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 00:50:13.279408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.280146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.280528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.280922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4145be15-77fe-42f7-9957-bdbd06f1aeb0 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 30 00:50:13.286009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:50:13.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application yield resource Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application next(self.gen) Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise self.value Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application session.commit() Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 00:50:13.293870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.session.flush() Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application compat.raise_( Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 00:50:13.294984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 00:50:13.296181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application conn.query(q) Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application result.read() Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 00:50:13.297389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [parameters: {'user_id': '7cee85cd8e08434994972ae3aecd99fc', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:13.298472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 00:50:13.299523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 496/974] 10.222.0.52 () {60 vars in 1131 bytes} [Tue Aug 30 00:50:13 2022] POST /identity/v3/users => generated 160 bytes in 241 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:13.300728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440babd-1162-4cd2-92e3-a160e54b4484 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.301309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.301405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440babd-1162-4cd2-92e3-a160e54b4484 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.301405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9440babd-1162-4cd2-92e3-a160e54b4484 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.301564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.301644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 497/975] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:13.301644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440babd-1162-4cd2-92e3-a160e54b4484 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5ca832bc-1c61-4c71-bed8-b4f9f78f40be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:13.313268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:13.314414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:13.316213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:13.322500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 479/976] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:13.322500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.322500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.322500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.342508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:13.343634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:13.379500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.421768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 480/977] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:13.421768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.444685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9440babd-1162-4cd2-92e3-a160e54b4484 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.493258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.494204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.495364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.495840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.505877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.507732 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.514455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:13.520837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:13.522122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 00:50:13.523115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:13.530199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 498/978] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:13.531687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.571168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7effdde7-c18f-4df5-b35a-ba21ac683407 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.610397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.611452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.612235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.613081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.614070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:13.614704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.616976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba679823-b082-4f74-b388-606ce2d001c0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.633172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 481/979] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:50:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:13.635829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5ed30-50c1-4cc2-9718-d97b54d0717b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.636373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5ed30-50c1-4cc2-9718-d97b54d0717b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.636816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5ed30-50c1-4cc2-9718-d97b54d0717b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.638786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 482/980] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:13.640205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5ed30-50c1-4cc2-9718-d97b54d0717b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.662997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece0510-7bc7-4b05-beb7-446972c65cf0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.678234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.699094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.699702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.699997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.700246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.700938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:13.701237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.702892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac73da-cf04-4bfd-afe1-965d6704427a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.749762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 499/981] 10.222.0.52 () {56 vars in 1101 bytes} [Tue Aug 30 00:50:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:13.749762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.749762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.749762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.765241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5ed30-50c1-4cc2-9718-d97b54d0717b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:13.773343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:13.775072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:13.786177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:13.820779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:13.821431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:13.821698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:13.822008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:13.822761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] RBAC: Adding query filter params (role.id=c578d26674f3470abc3ec8613c828c95, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=3002491f2c61493890d2e4272e9ba522) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:13.823718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:13.825196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:13.841775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 483/982] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:13 2022] GET /identity/v3/role_assignments?user.id=3002491f2c61493890d2e4272e9ba522&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=c578d26674f3470abc3ec8613c828c95 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:13.841775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:13.876669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 500/983] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:13.966339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d8ed2f-3d4b-4f5a-bc1d-a185e664df96 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.009782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.010495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.010777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.011069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.011645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.013012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:14.023523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:14.024568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:14.026572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 484/984] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:14.031358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:14.139358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd1350e-23c2-43f5-bb1e-389f8762afb5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.140817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:14.158255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:14.175537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.176346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.176906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.177720 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.178449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:14.178826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.180266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:14.190586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 501/985] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:14.191611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:14.211132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:14.275486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97060e50-51c1-48df-9adf-b31ed2f7f2e6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.303040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888fbb5-34ad-47fb-b6d0-21e048bcc7db None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.313638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.314369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.314724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.315070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.315805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.317244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:14.341386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.342095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.342373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.342644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.343269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:14.343570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.344938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:14.350379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 502/986] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:14.356381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:14.386823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:14.401868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 485/987] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:14 2022] POST /identity/v3/services => generated 242 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:14.475639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd969a-9360-4ad5-8a9c-bf0fa6cb56d4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.524442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.525077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.525355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.525628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.526540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] RBAC: Adding query filter params (interface=admin, service_id=824450beaea94868a523052eddceb74d, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:14.527397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.528808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97d5f1-fe63-4bbf-89f4-dceb7ea69ebf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:14.539292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 503/988] 10.222.0.52 () {56 vars in 1249 bytes} [Tue Aug 30 00:50:14 2022] GET /identity/v3/endpoints?service_id=824450beaea94868a523052eddceb74d&interface=admin®ion_id=RegionOne => generated 193 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:14.540580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:14.648586 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:14.652853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f556eaef-f949-41ec-9052-dd739bf58368 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:14.689644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740281cd-c2f2-4854-828b-5e7162021fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:14.690396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740281cd-c2f2-4854-828b-5e7162021fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:14.690750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740281cd-c2f2-4854-828b-5e7162021fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:14.691099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740281cd-c2f2-4854-828b-5e7162021fe3 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:14.691780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740281cd-c2f2-4854-828b-5e7162021fe3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:14.693346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740281cd-c2f2-4854-828b-5e7162021fe3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:14.700298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 486/989] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:50:14 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:50:15.856062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6194c-aeb4-40cb-943f-5348adea08d7 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:15.856680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6194c-aeb4-40cb-943f-5348adea08d7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:15.856680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6194c-aeb4-40cb-943f-5348adea08d7 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:15.858140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 504/990] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:15.863197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57f2d9-0e51-4b1d-b80c-2f7ec737dc3f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:15.863609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57f2d9-0e51-4b1d-b80c-2f7ec737dc3f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:15.863827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57f2d9-0e51-4b1d-b80c-2f7ec737dc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:15.882035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8c57f2d9-0e51-4b1d-b80c-2f7ec737dc3f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:15.883237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8c57f2d9-0e51-4b1d-b80c-2f7ec737dc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:15.958765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 487/991] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:16.231842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6194c-aeb4-40cb-943f-5348adea08d7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:16.337213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:16.341826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566355b-8560-4964-99ee-0bbef7c9a205 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.342123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566355b-8560-4964-99ee-0bbef7c9a205 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.342396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566355b-8560-4964-99ee-0bbef7c9a205 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.343735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 488/992] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:16.348950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.349233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.349511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.350196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6194c-aeb4-40cb-943f-5348adea08d7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:16.368308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:16.370075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:16.391919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:16.392528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.393675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.393675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] PATH_INFO: `/v3/services/image` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.393935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:16.395143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:16.404705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:16.406054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:16.407079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 505/993] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Aug 30 00:50:16 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:16.411656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:16.454181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 489/994] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:16.526863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6650075-2123-4eca-af9c-d4b016d477ef None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:16.563714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:16.564329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.564612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.564890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.565517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] RBAC: Adding query filter params (name=image) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:16.565851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:16.567223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd76359-b21c-4e97-b3ae-8a18e329cb65 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:16.572280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 506/995] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:16.578096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:16.605278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:16.631216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e664f6-b7e9-45e9-8943-20c8f01abdb3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.631508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e664f6-b7e9-45e9-8943-20c8f01abdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.631787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e664f6-b7e9-45e9-8943-20c8f01abdb3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.633246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 507/996] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:16.638795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.639082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.639376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.663473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:16.664694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:16.683074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:16.697489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e329ab7f-fa48-4ae0-9a16-e33bb185a0bb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:16.736142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:16.737055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.737055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.737357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.738032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:16.738341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:16.739715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:16.741219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 508/997] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:16.742224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:16.744682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 490/998] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:16 2022] GET /identity/v3/services?type=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:16.881759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957e51ef-c605-4e77-ae63-f692b327cfd9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:16.930375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:16.931034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:16.931317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:16.931590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] PATH_INFO: `/v3/services/compute` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:16.932644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:16.934240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:16.943556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:16.944567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:16.945446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 509/999] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:16.950507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.019720 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.035832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:17.044813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:17.057341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb88862-f7ac-4c00-bb5e-7bb9165f6272 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.096609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.098063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.098351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.098631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.099266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:17.099573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.101466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.108468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 491/1000] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:17.113818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.130500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6a9284-2392-4a1e-a981-5d5ed7f613bf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.172820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.173590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.173912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.174192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.174769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.176149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:17.186115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:17.187192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:17.188043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 510/1001] 10.222.0.52 () {56 vars in 1126 bytes} [Tue Aug 30 00:50:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:17.193148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.225488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e65e70-68b8-4c95-82c7-fd40870a76c6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.264250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.264882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.265167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.265451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.266120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:17.266428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.267980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.274407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 492/1002] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:17.281245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.296607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:17.302762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:17.327446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9795bc17-0b7c-4992-af7f-4394772b3791 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.365351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.365999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.366444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.366552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.367174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:17.367475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.368860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.377797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 511/1003] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:17.380633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.397446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a672b-f856-440b-9442-788fb4781441 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.437424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.438066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.438359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.438633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.439336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] RBAC: Adding query filter params (interface=public, service_id=15221eac7e0540e7810d44582a398cc8, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:17.439648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.441259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.446861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 493/1004] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:50:17 2022] GET /identity/v3/endpoints?service_id=15221eac7e0540e7810d44582a398cc8&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:17.452810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.495696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca81144-5064-457a-9666-3f0e58f507a7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.532936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.533468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.533842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.534052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.534678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:17.534982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.536360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.542218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 512/1005] 10.222.0.52 () {56 vars in 1122 bytes} [Tue Aug 30 00:50:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:17.543463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:17.548048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:17.551679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:17.565583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b76f06d-938e-4dd9-a9e3-4469b6e6cd60 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.627626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50547b-7fb3-493b-859f-a5789a65bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.629052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50547b-7fb3-493b-859f-a5789a65bb24 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.629559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50547b-7fb3-493b-859f-a5789a65bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.630086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50547b-7fb3-493b-859f-a5789a65bb24 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.631205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50547b-7fb3-493b-859f-a5789a65bb24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.633754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d50547b-7fb3-493b-859f-a5789a65bb24 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.657772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 494/1006] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:50:17 2022] GET /identity/v3/services => generated 1727 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 00:50:17.673082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25732ca7-7856-443e-ab65-f96f0c184096 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:17.713031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfab5f-c79e-4c5e-bc97-60f62b3e4e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:17.714229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfab5f-c79e-4c5e-bc97-60f62b3e4e6f None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:17.714507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfab5f-c79e-4c5e-bc97-60f62b3e4e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:17.714778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfab5f-c79e-4c5e-bc97-60f62b3e4e6f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:17.715498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfab5f-c79e-4c5e-bc97-60f62b3e4e6f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:17.716913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfab5f-c79e-4c5e-bc97-60f62b3e4e6f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:17.817805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 513/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:50:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:18.708725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44187cc9-1e99-4313-a1b3-4d5027c67903 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:18.709242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44187cc9-1e99-4313-a1b3-4d5027c67903 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:18.709361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44187cc9-1e99-4313-a1b3-4d5027c67903 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:18.710785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 495/1008] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:18.716853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d10ef8-ddd9-493f-999a-1c203230ba82 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:18.717315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d10ef8-ddd9-493f-999a-1c203230ba82 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:18.717436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d10ef8-ddd9-493f-999a-1c203230ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:18.736180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e4d10ef8-ddd9-493f-999a-1c203230ba82 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:18.737441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e4d10ef8-ddd9-493f-999a-1c203230ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:18.806005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:18.821784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 514/1009] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:18 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:19.102270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44187cc9-1e99-4313-a1b3-4d5027c67903 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:19.212045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44187cc9-1e99-4313-a1b3-4d5027c67903 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:19.249892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32292747-9aca-4b45-a668-12d050460e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:19.250507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292747-9aca-4b45-a668-12d050460e5e None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.250791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292747-9aca-4b45-a668-12d050460e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.251067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32292747-9aca-4b45-a668-12d050460e5e None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.251729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292747-9aca-4b45-a668-12d050460e5e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:19.253170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32292747-9aca-4b45-a668-12d050460e5e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:19.343513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 496/1010] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:19 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:19.620670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4398-a612-426c-b9da-6ba39864bcae None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.621052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4398-a612-426c-b9da-6ba39864bcae None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.621254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c4398-a612-426c-b9da-6ba39864bcae None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.622661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 515/1011] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:19.627933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbfa75-8cd5-4b4c-92a7-c1062cb988b3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.628225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbfa75-8cd5-4b4c-92a7-c1062cb988b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.628510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cbfa75-8cd5-4b4c-92a7-c1062cb988b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.636576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:19.649390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-74cbfa75-8cd5-4b4c-92a7-c1062cb988b3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:19.650848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-74cbfa75-8cd5-4b4c-92a7-c1062cb988b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:19.729396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 497/1012] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:19.786926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d860bd-64e7-4390-a542-1adf8ebf168e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.787434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d860bd-64e7-4390-a542-1adf8ebf168e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.787434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d860bd-64e7-4390-a542-1adf8ebf168e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.788842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 516/1013] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:19.794752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:19.795045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:19.795323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:19.814777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:19.814777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:19.883597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 498/1014] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:19 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:19.998551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d860bd-64e7-4390-a542-1adf8ebf168e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.108304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d860bd-64e7-4390-a542-1adf8ebf168e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.143906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.144731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.145153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.145591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] PATH_INFO: `/v3/services/compute` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.146461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.147993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.154626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:20.157784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:20.158798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:20.159652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 517/1015] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:20.164641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.219733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:20.226634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:20.262814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4d9b3-63fe-44f6-8bf4-def0ae3e1246 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.273586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296efa5-340f-4345-b2fe-2e0fb16e0aba None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.298834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.299501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.299786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.300064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.301177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.302644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.304617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 499/1016] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:20.311253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.313414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.314064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.314351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.314634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.315270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] RBAC: Adding query filter params (name=compute) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:20.315579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.316962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.322302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 518/1017] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:20.328265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.451359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca5e0b-0cad-40f7-9e41-f7af1c835884 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.467454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894a0697-3381-45b1-a06c-99f2a388d08e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.490194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:20.491115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:20.499310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69672dbf-a1bd-47a5-8dd5-f3ee55be340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.499926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69672dbf-a1bd-47a5-8dd5-f3ee55be340a None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.500207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69672dbf-a1bd-47a5-8dd5-f3ee55be340a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.500570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69672dbf-a1bd-47a5-8dd5-f3ee55be340a None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.501134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69672dbf-a1bd-47a5-8dd5-f3ee55be340a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.502576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69672dbf-a1bd-47a5-8dd5-f3ee55be340a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.509176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.509830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.510699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.521566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.521566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:20.521566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.521566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.521566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 519/1018] 10.222.0.52 () {56 vars in 1112 bytes} [Tue Aug 30 00:50:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:20.524612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:20.612784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 500/1019] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:20 2022] POST /identity/v3/projects => generated 311 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:20.634452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79214413-48dc-4796-8de4-1d7186e26522 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:20.673645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e21665-fa4a-4597-b2df-c9edc832a98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:20.674303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e21665-fa4a-4597-b2df-c9edc832a98e None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:20.674589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e21665-fa4a-4597-b2df-c9edc832a98e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:20.674867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e21665-fa4a-4597-b2df-c9edc832a98e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:20.675550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e21665-fa4a-4597-b2df-c9edc832a98e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:20.676978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e21665-fa4a-4597-b2df-c9edc832a98e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:20.767004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 520/1020] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:50:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:21.307738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc07b9-57de-4cc2-9baa-4736751d52ed None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:21.308262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc07b9-57de-4cc2-9baa-4736751d52ed None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:21.308341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc07b9-57de-4cc2-9baa-4736751d52ed None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:21.309671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 501/1021] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:21.315064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:21.315534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:21.315614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:21.334868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:21.336067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:21.368934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:21.415801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 521/1022] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:21.683782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc07b9-57de-4cc2-9baa-4736751d52ed None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:21.796500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc07b9-57de-4cc2-9baa-4736751d52ed None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:21.832386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:21.833029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:21.833313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:21.833584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:21.834801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:21.836515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:21.846435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:21.847548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:21.848596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:21.850213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 502/1023] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Aug 30 00:50:21 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:21.853485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:21.959104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6caad5-a253-457b-a4d8-35a5c4036c53 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:21.996880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:21.997500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:21.997806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:21.998078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:21.998706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] RBAC: Adding query filter params (name=image) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:21.999015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:22.000390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:22.012232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 522/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:22.012383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:22.063323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:22.121206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88c8a1-73b6-4068-88e9-225dafffedc1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:22.158234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:22.158877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.159158 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.159436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.160060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:22.160362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:22.161900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:22.167408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 503/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:22.173696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:22.202521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:22.279363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c755d84-9b66-4cf1-923f-91ad0d2aee4d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:22.315453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:22.316121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.316374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.316605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.317298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] RBAC: Adding query filter params (interface=public, service_id=1ed40eb816f246bb9f07884256592e6f, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:22.317603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:22.319002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b627a-6444-4b5f-8423-94a30bdb6629 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:22.329342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 523/1026] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:50:22 2022] GET /identity/v3/endpoints?service_id=1ed40eb816f246bb9f07884256592e6f&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:22.332602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:22.442391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3684a751-9afd-46fb-9925-20141ff5e898 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:22.453332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:22.485052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533cfef-5204-4d46-b232-aecd38eb252b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:22.485052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533cfef-5204-4d46-b232-aecd38eb252b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.485052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533cfef-5204-4d46-b232-aecd38eb252b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.485052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533cfef-5204-4d46-b232-aecd38eb252b None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.485491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533cfef-5204-4d46-b232-aecd38eb252b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:22.485807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533cfef-5204-4d46-b232-aecd38eb252b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:22.497966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 504/1027] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:50:22 2022] GET /identity/v3/services => generated 1957 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:50:22.629545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b806b3-f1bc-4155-b695-0b27bf2f24c7 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.629985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b806b3-f1bc-4155-b695-0b27bf2f24c7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.630190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b806b3-f1bc-4155-b695-0b27bf2f24c7 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.631484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 524/1028] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:22.638244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25e34-6905-4cae-aceb-f238a0f75028 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.638512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25e34-6905-4cae-aceb-f238a0f75028 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.638786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25e34-6905-4cae-aceb-f238a0f75028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.658228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-13a25e34-6905-4cae-aceb-f238a0f75028 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:22.659430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-13a25e34-6905-4cae-aceb-f238a0f75028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:22.731966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 505/1029] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:22.820052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b79e3-ba09-4415-845e-36dd91a86cd5 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.820381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b79e3-ba09-4415-845e-36dd91a86cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.820594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b79e3-ba09-4415-845e-36dd91a86cd5 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.822098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 525/1030] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:22.827296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:22.827586 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:22.827860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:22.846799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:22.847982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:22.920420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 506/1031] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:22 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:23.010192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b79e3-ba09-4415-845e-36dd91a86cd5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.108775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.119727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b79e3-ba09-4415-845e-36dd91a86cd5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.155547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.156174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.156452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.156726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.157881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:23.159315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:23.161157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 526/1032] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:23.166594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.195446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.199923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.275291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bda623-5a42-49be-ab16-e42338aa7da7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.313206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.313855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.314136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.314412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.315079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:23.315295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b353593-1429-44b0-96fc-d45fd70a745e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.316531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:23.354472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.355103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.355454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.356059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.366730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:23.371754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:23.373985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:23.375190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 527/1033] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:23.378726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.403933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.459547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 507/1034] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:50:23 2022] POST /identity/v3/users => generated 285 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:23.517392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfad651-7d91-485d-aaae-c75b0e79aec7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.560820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.561447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.561758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.562033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.562709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:23.563014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:23.564414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:23.574534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 528/1035] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:23.580164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.680471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.693245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0de78d-578c-4b91-a22b-49ce65e9d77f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.731717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.731717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.731717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.731717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.742739 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:23.744730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:23.762229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:23.763351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:23.764307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:23.764307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:23.764307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:23.764307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 508/1036] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:23.769397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:23.866430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:23.885541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff8921-3324-4ca1-99af-129ba74b20a1 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:23.921672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:23.922329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:23.922851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:23.923032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:23.923515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:23.923830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:23.925190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:23.942833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 529/1037] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:23.944522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:24.052253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296702f-ca31-4332-b7b1-71f048f0348c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:24.088339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:24.088984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.089266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.089545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.097398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:24.098895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:24.101588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:24.111785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:24.121505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:24.122494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 509/1038] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:50:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:24.123120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:24.243998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09463a-8884-4e73-87fa-529ddb00b457 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:24.286957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:24.287895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.288272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.288653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.289653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:24.290117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:24.292159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1c4d1-5c1c-435c-bb7f-2df2d52ed8b0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:24.314313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:24.324848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 530/1039] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:50:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:24.326804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:24.445963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de11a6cc-2f03-4be3-b0a6-bfc7447ed2a5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:24.491405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:24.492262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.492262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.492581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.507216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f40e-27cf-427f-919d-ea7ffa694668 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.507731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f40e-27cf-427f-919d-ea7ffa694668 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.507974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc59f40e-27cf-427f-919d-ea7ffa694668 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.510621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 531/1040] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:24.516281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.516596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.516848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.536422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:24.537760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:24.578045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:24.581065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:24.582760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:24.614960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 532/1041] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:24 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:24.694973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 510/1042] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:24 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:50:24.702623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:24.808917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d35aa-38e1-4939-b07b-bc3537e1eb4a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:24.839165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:24.845551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:24.846222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:24.853058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:24.853058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:24.858331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:24.859790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:24.875345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:24.876349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:24.877276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:24.877276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:24.877276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:24.881321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 533/1043] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:24.882366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:24.887033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:24.998259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.001418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f66f9-a06c-46f5-a40a-31fad4cf9916 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.035261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc87cd0d-3b2f-42a8-b910-1af0e471f136 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.039501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.040119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.040408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.040668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.041397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:25.041813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.043044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.054722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 511/1044] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:25.061608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.086263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.086899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.087180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.087711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] PATH_INFO: `/v3/services/image` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.088975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.090589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:25.099475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:25.105705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:25.106668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 534/1045] 10.222.0.52 () {56 vars in 1105 bytes} [Tue Aug 30 00:50:24 2022] GET /identity/v3/services/image => generated 86 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:25.107834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.163275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.180817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8546b4c7-1706-439a-b830-43b23fc90ad5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.218279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.218926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.219209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.219492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.221577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda78d6d-5216-4796-bdbb-21faacfaffda None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.226413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.227864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:25.240268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:25.241277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:25.242197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:25.246640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 512/1046] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:25.247792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.258556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.259173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.259528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.259876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.260684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] RBAC: Adding query filter params (name=image) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:25.261000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.262483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.268031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 535/1047] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:25.272812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.273472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.355805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6049236-7bcd-408c-baa3-9b6488526a92 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.390026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3872711c-23ca-4f9d-a7d5-c80ac786ba09 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.393574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.394235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.394525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.394805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.395514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:25.395829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.397718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.401437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.418403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 513/1048] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:25.419700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.431823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.432438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.432716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.432976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.433602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] RBAC: Adding query filter params (type=image) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:25.433927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.435306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.441415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 536/1049] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/services?type=image => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:25.446987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:25.520337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:25.540426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b730709-ba81-4459-96b9-8214e2b6ce5f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.566375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7fd63a-a5ae-402e-b6d5-465dc79c8a9b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:25.580663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.581276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.581563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.581878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.585069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] RBAC: Adding query filter params (role.id=c578d26674f3470abc3ec8613c828c95, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=e1a623f1fc434449a76106e74756e35c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:25.585396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.586889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116e631-320d-491f-a403-19d56e552e38 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.593118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 514/1050] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/role_assignments?user.id=e1a623f1fc434449a76106e74756e35c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=c578d26674f3470abc3ec8613c828c95 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:25.595063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b9f17-d19f-4166-847a-61132d93a180 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.595355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b9f17-d19f-4166-847a-61132d93a180 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.595631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b9f17-d19f-4166-847a-61132d93a180 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.598003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 515/1051] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:25.603319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.604316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.604617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.613004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:25.613797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:25.614206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:25.614626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:25.615698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:25.618038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:25.626223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:25.627473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:25.705707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 516/1052] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:25 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:25.774707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 537/1053] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:50:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:25.981898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.061404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.089991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554af7b-25ed-4b39-ad06-7feac892f52c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.131866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.132505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.132784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.133057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.143563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:26.148315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:26.149286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:26.150180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 517/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:26.156104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.189338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.263448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e2035-8834-4198-a144-7efd2a9db773 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.298178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.298793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.299073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.299348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.300032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:26.300337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.301714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.310701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 538/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:26.316589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.424455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d024b59-2e1a-4a47-bca6-efafa4ca26ea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.454577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.460890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.461529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.461837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.462113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] PATH_INFO: `/v3/users/ed0b5222d73543a189dc44d1c3fa5c8b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.478370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0b5222d73543a189dc44d1c3fa5c8b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.479870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.487008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 518/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/users/ed0b5222d73543a189dc44d1c3fa5c8b => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:26.488384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.582638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.593263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c77cc-87ed-4b6d-9720-61ed8a937620 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.629134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.629800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.630087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.630353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.701148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.702639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.797630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:26.812655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 539/1057] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:26.813995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:26.923286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11fc7e6-964f-4337-b5d6-47eccb128019 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:26.959632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981414b0-e359-41a5-9a54-a47b75bc4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:26.960266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981414b0-e359-41a5-9a54-a47b75bc4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:26.960546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981414b0-e359-41a5-9a54-a47b75bc4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:26.960822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981414b0-e359-41a5-9a54-a47b75bc4bc7 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:26.961974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981414b0-e359-41a5-9a54-a47b75bc4bc7 None admin] RBAC: Authorizing `identity:get_project(project_id=8fef9c4ec38843c98a393872d0a06b56)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:26.964356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981414b0-e359-41a5-9a54-a47b75bc4bc7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:26.966231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 519/1058] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:26.972272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:27.077780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae5f173-1961-459a-928f-ac09a08afd9c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:27.113231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:27.113947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:27.114217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:27.114488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:27.115259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8fef9c4ec38843c98a393872d0a06b56, user.id=ed0b5222d73543a189dc44d1c3fa5c8b) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:27.115566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:27.117087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5217e20-d29a-46c2-b4eb-a165ff0ebb6a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:27.129670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 540/1059] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:26 2022] GET /identity/v3/role_assignments?user.id=ed0b5222d73543a189dc44d1c3fa5c8b&scope.project.id=8fef9c4ec38843c98a393872d0a06b56&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:27.586286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fb19d-dff2-4bbd-98be-970d19d285f4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:27.586638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fb19d-dff2-4bbd-98be-970d19d285f4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:27.586853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fb19d-dff2-4bbd-98be-970d19d285f4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:27.589339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 520/1060] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:27.594457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ba36-e5fa-4797-a6d5-da6e1495b6cc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:27.594923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ba36-e5fa-4797-a6d5-da6e1495b6cc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:27.595023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe01ba36-e5fa-4797-a6d5-da6e1495b6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:27.614150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fe01ba36-e5fa-4797-a6d5-da6e1495b6cc None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:27.615348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fe01ba36-e5fa-4797-a6d5-da6e1495b6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:27.674276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:27.691345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 541/1061] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:27.752964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4a7d-bc40-4f57-bd0a-a83bbba07399 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:27.753250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4a7d-bc40-4f57-bd0a-a83bbba07399 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:27.753460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d4a7d-bc40-4f57-bd0a-a83bbba07399 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:27.754983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 521/1062] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:27.760082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:27.760375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:27.760652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:27.779019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:27.780212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:27.851032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 542/1063] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:27.958967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d4a7d-bc40-4f57-bd0a-a83bbba07399 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.022455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.068162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d4a7d-bc40-4f57-bd0a-a83bbba07399 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.106500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.107137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.107414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.107684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.118462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:28.123955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:28.124918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:28.126628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 522/1064] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 30 00:50:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:28.128626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.131326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.243763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b59e76d-bbee-4105-98f8-84461bd3f51c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.250908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eca6a2-75c0-4005-8f4c-ecc40e4dc2d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.262033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.269479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.281661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.282237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.282518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.282804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.295104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.295449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.295720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.295994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.296650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:28.297131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.298413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.307141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 523/1065] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:28.312590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.364275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.365766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.426006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4d10b3-2e06-46ea-a539-1d073e9268b3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.463598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.464233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.464518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.464796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.476183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.477631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.489990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 543/1066] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:28.496198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:28.499103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:28.500023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:28.505817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:28.505817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:28.505817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:28.505817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 524/1067] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:28.506043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.514684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.539819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.602199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b6816-5b47-4c13-bc20-3045f0231f04 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.623093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a61fd62-c7e5-4a86-b13e-8967ccdd2aa5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.642057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.642553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.642834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.643098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.654481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.655963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.663291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.664622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.665136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.666780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.667767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:28.668088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.669478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:28.673186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:28.674279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:28.675253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:28.675253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:28.675253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:28.679776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 544/1068] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:28.680317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.687504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 525/1069] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:28.688208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:28.761255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.788542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:28.799415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef6962-964c-4821-a9e8-9936c1af21a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.801918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d6a58-da80-4f38-b4a1-71b7ea0d4fb4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:28.839186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.839806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.840188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.840370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.841109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:28.841420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.842699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:28.842823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.843649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:28.844412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:28.844874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:28.855545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 545/1070] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:28.856054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:28.857553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0123c4ed-caef-4f3d-81f8-383fae5f132d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:28.880485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:28.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:28.888573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 526/1071] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:28.889316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.022050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dc9fa-8b13-4780-847c-6bb3496e96d2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.053951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.061829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184384e-d55f-40a6-97a3-15131573d8d4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.062345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184384e-d55f-40a6-97a3-15131573d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.062435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184384e-d55f-40a6-97a3-15131573d8d4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.063705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 527/1072] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:29.064688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.065303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.065592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.065905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.066644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:29.071032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.071032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.071205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.071205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.071205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.090380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 546/1073] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:50:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:29.091234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:29.092461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:29.096466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.178391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 528/1074] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:50:29.213399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327e3d06-1ef2-4974-bbbb-357671120f09 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.247684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.248399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.248686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.248957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.258958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.261014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:29.275200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:29.276414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:29.277595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:29.277595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:29.277595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:29.278036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 547/1075] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:29.282250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.367179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.386844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb52fc-3df3-4f05-be17-81d2265e176a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.427030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.427647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.427930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.428205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.428852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:29.429155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.430553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.442705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 529/1076] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:29.448485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.451036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.493966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.555212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31abca-b22a-417a-a70a-dffbe50b045a None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.558078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2dd036-1038-41a4-ad9f-a8c6b13e6a43 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.591092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.591727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.592021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.592298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.594129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.594762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.595048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.595326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.599224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.600712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.603043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.606510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:29.611412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:29.612537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:29.620595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 530/1077] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:29.621325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:29.622536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:29.623435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 548/1078] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:29.623926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.624026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.719750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.723283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd51cc-fea1-4316-8482-363b58f2cb67 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.724994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48187148-541c-49bb-a2cc-bf137eef5bb4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.757610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.758254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.758646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.758937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.760320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:29.760596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.760691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.761292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.761613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.761954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.762072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.762641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:29.762976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.764354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.773248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 531/1079] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:29.774324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 549/1080] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:29.779555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.780412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:29.844038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:29.881307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372c16d-ac68-4c0f-b7ce-324b3e99d424 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.883420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54540c90-a56c-4db5-8ed6-1f360f211645 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:29.915631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.916446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.916762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.917036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/e1a623f1fc434449a76106e74756e35c/roles/c578d26674f3470abc3ec8613c828c95` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.919224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=e1a623f1fc434449a76106e74756e35c, role_id=c578d26674f3470abc3ec8613c828c95)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.919659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:29.920311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:29.920632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:29.920723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.920962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] PATH_INFO: `/v3/users/ed0b5222d73543a189dc44d1c3fa5c8b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:29.922221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] RBAC: Authorizing `identity:get_user(user_id=ed0b5222d73543a189dc44d1c3fa5c8b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:29.923680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:29.925297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 532/1081] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/users/ed0b5222d73543a189dc44d1c3fa5c8b => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:29.931042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:30.002160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 550/1082] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:29 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/e1a623f1fc434449a76106e74756e35c/roles/c578d26674f3470abc3ec8613c828c95 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:50:30.039762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15690bb-b619-44c0-86f9-c697caffc80b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:30.075556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:30.082158 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d969-e0c7-4e87-bc3b-8d5b1ae403d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:30.082875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d969-e0c7-4e87-bc3b-8d5b1ae403d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:30.083182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d969-e0c7-4e87-bc3b-8d5b1ae403d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:30.083397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d969-e0c7-4e87-bc3b-8d5b1ae403d3 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:30.084651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d969-e0c7-4e87-bc3b-8d5b1ae403d3 None admin] RBAC: Authorizing `identity:get_project(project_id=8fef9c4ec38843c98a393872d0a06b56)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:30.086335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b87d969-e0c7-4e87-bc3b-8d5b1ae403d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:30.089075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 533/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:29 2022] GET /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:30.094769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:30.116091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:30.205129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe199e-5b4f-4bcc-8d19-468802bd3291 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:30.240024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342254c3-cc57-46c4-b53e-9b4e55eabf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:30.240694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342254c3-cc57-46c4-b53e-9b4e55eabf97 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:30.240970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342254c3-cc57-46c4-b53e-9b4e55eabf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:30.241245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342254c3-cc57-46c4-b53e-9b4e55eabf97 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56/users/ed0b5222d73543a189dc44d1c3fa5c8b/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:30.243292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342254c3-cc57-46c4-b53e-9b4e55eabf97 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fef9c4ec38843c98a393872d0a06b56, user_id=ed0b5222d73543a189dc44d1c3fa5c8b, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:30.245296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342254c3-cc57-46c4-b53e-9b4e55eabf97 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:30.322837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 551/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:30 2022] PUT /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56/users/ed0b5222d73543a189dc44d1c3fa5c8b/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:50:31.968965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fbc1e-3155-4341-bba7-7ecf1191eba0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.969523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fbc1e-3155-4341-bba7-7ecf1191eba0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.969523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315fbc1e-3155-4341-bba7-7ecf1191eba0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.970919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 534/1085] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:31.976686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd232e6-83d1-4115-a23e-80007d282f3c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:31.977107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd232e6-83d1-4115-a23e-80007d282f3c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:31.977245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd232e6-83d1-4115-a23e-80007d282f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:31.996431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ecd232e6-83d1-4115-a23e-80007d282f3c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:31.997642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ecd232e6-83d1-4115-a23e-80007d282f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:32.065657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 552/1086] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:32.254762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b8ea-bec7-4443-a25c-921ba5ffafbe None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.254978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b8ea-bec7-4443-a25c-921ba5ffafbe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.255263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453b8ea-bec7-4443-a25c-921ba5ffafbe None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.256600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 535/1087] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:32.262737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.263120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.263479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.281401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:32.282646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:32.284623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.338051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b8ea-bec7-4443-a25c-921ba5ffafbe None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.350734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 553/1088] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:32.577176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453b8ea-bec7-4443-a25c-921ba5ffafbe None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.613086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.613717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.614031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.614315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.624409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:32.626192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:32.629653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:32.630631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:32.631656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 536/1089] 10.222.0.52 () {56 vars in 1120 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:32.636804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.698763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.726624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8a33d-81ff-4c6a-9b85-ce6e950dd4ac None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.743358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd6ead-ece5-42ef-bed1-df62c98e6acc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.762949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.763599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.763886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.764162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.771111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.776165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:32.779279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.779887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.780171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.780450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.781112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:32.781225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:32.782331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:32.783367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:32.784122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.784205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 554/1090] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:32.787792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.788763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.802512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 537/1091] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:32.804162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.892688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247cbf7a-97c1-4245-a627-762e22ab1265 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.908484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeae2bcc-baf5-4633-b852-9057e9a98114 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:32.928152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.928738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:32.928865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.929417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.929696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.930402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:32.930702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.932621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.941110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 555/1092] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:32.944270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:32.944895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:32.945173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:32.945442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:32.947051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:32.956997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:32.958488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:32.973984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:32.975604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:32.976803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:32.976803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:32.976803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:32.976803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 538/1093] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:32.981585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.023029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.070287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae77772-07b3-4ca0-882b-058090bc5423 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.100545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf6069-e5dc-4238-978e-370f64f362ce None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.104939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.105497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.105804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.106083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] PATH_INFO: `/v3/users/ed0b5222d73543a189dc44d1c3fa5c8b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.107231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] RBAC: Authorizing `identity:get_user(user_id=ed0b5222d73543a189dc44d1c3fa5c8b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.108661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.110278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 556/1094] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/users/ed0b5222d73543a189dc44d1c3fa5c8b => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:33.116060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.135729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.136337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.136615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.136888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.137521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:33.137854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.139210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.151573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 539/1095] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:33.158934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.190668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.223433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c04e36-9797-4c46-8da4-b0192ba70de9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.251282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.259292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.260004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.260299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.260574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.262371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914e3fd-7f14-482e-96ce-9d1d721a910c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.298099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.298737 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.299022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.299301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.307412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.309137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:50:33.321187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:33.322412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:33.323584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 540/1096] 10.222.0.52 () {56 vars in 1093 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:50:33.328417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.330779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.332253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.427308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.437031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9e63f5-5419-4bb4-8a38-a04b73d9fe91 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.437479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 557/1097] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:50:33.443466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.472770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.474687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.475307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.475610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.475878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.476610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:33.476929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.478990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.497146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 541/1098] 10.222.0.52 () {56 vars in 1133 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:33.502992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.549267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49fe10-33f8-44f7-be0a-10e7412bbc0e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.583301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.583936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.584228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.584508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.585632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] RBAC: Authorizing `identity:get_project(project_id=8fef9c4ec38843c98a393872d0a06b56)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.587364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.589569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 558/1099] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:33.595877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.605175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6486e8-27ce-4e5d-818a-4abca77bda68 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.639264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.639953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.640200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.640446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.660567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.692337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:33.698097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16824656-40a9-49e2-85aa-f3a350cf4445 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.711925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.714549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.733406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.734052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.734330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.734601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.735320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8fef9c4ec38843c98a393872d0a06b56, user.id=ed0b5222d73543a189dc44d1c3fa5c8b) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:33.735624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.737054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:33.742752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 559/1100] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/role_assignments?user.id=ed0b5222d73543a189dc44d1c3fa5c8b&scope.project.id=8fef9c4ec38843c98a393872d0a06b56&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:50:33.826944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 542/1101] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 00:50:33.833138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:33.944673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e13efbf-4d9d-426d-9362-ac8a78f18a7f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:33.980621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:33.981254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:33.981539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:33.981834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:33.992307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:33.993884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:50:34.009972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:34.011159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:50:34.012185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:50:34.012185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:50:34.012185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:34.012185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 560/1102] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:34.017091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:34.067805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:34.118527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add35825-5b9e-45bd-a722-78735b5e7375 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:34.152714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:34.153323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:34.153602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:34.153905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:34.154538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:34.154842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:34.156202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:34.169523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 543/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:50:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:34.175323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:34.191260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:34.278707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f5c531-ef48-470d-bb98-29d11149a298 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:34.313905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:34.314536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:34.314820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:34.315095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:34.321782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:34.323289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:50:34.334414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:34.336733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:50:34.338761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 561/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:50:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:34.342431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:34.446295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6beb3c6-e7be-48fe-bbe8-4b99995994c8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:34.457299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:34.482555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:34.483171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:34.483445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:34.483714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:34.484394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:34.484696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:34.486157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb11e12-c751-4461-99ac-60bd463f80b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:34.498074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 544/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:50:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:34.504407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:34.574979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:34.604963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962508e9-e382-4ecc-bfc9-a14b42a0bfdd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:34.639885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:34.640495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:34.640779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:34.641054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:34.641805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] RBAC: Adding query filter params (role.id=c578d26674f3470abc3ec8613c828c95, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=e1a623f1fc434449a76106e74756e35c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:34.642119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:34.643616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a7add-7af2-41c2-90ad-547b79087085 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:34.649478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 562/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:34 2022] GET /identity/v3/role_assignments?user.id=e1a623f1fc434449a76106e74756e35c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=c578d26674f3470abc3ec8613c828c95 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:35.709718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c75d36-3486-4e8a-a443-88109de4698a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:35.710290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c75d36-3486-4e8a-a443-88109de4698a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:35.710290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c75d36-3486-4e8a-a443-88109de4698a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:35.711684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 545/1107] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:35.717215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:35.717637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:35.717787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:35.735759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:35.736993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:35.804653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 563/1108] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:36.074616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c75d36-3486-4e8a-a443-88109de4698a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:36.175545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c75d36-3486-4e8a-a443-88109de4698a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:36.209218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b84b44e-1d50-4a1e-a7d1-dee738097bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:36.209872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b44e-1d50-4a1e-a7d1-dee738097bbe None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:36.210154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b44e-1d50-4a1e-a7d1-dee738097bbe None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:36.210430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b84b44e-1d50-4a1e-a7d1-dee738097bbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:36.211537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b44e-1d50-4a1e-a7d1-dee738097bbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:36.213796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b84b44e-1d50-4a1e-a7d1-dee738097bbe None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:36.215396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 546/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:36.221106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:36.285721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:36.319370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8070004c-2950-470b-b3c0-01f7c50b5698 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:36.354612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab823ef5-c813-4162-ad42-b1dddc0821c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:36.355144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab823ef5-c813-4162-ad42-b1dddc0821c4 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:36.355428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab823ef5-c813-4162-ad42-b1dddc0821c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:36.355701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab823ef5-c813-4162-ad42-b1dddc0821c4 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:36.356368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab823ef5-c813-4162-ad42-b1dddc0821c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:36.357872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab823ef5-c813-4162-ad42-b1dddc0821c4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:36.562749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 564/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:50:36 2022] POST /identity/v3/users => generated 286 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:38.519407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a685c-9a7c-4e7e-8634-d703346fff82 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:38.519897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a685c-9a7c-4e7e-8634-d703346fff82 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:38.520013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a685c-9a7c-4e7e-8634-d703346fff82 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:38.521402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 547/1111] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:38.527323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:38.527717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:38.527897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:38.548421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:38.549655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:38.616868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 565/1112] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:38.888572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a685c-9a7c-4e7e-8634-d703346fff82 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:38.929803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:38.991260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a685c-9a7c-4e7e-8634-d703346fff82 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:39.025304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:39.025957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:39.026242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:39.026518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] PATH_INFO: `/v3/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:39.027903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] RBAC: Authorizing `identity:get_role(role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:39.029280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:39.030641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 548/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:38 2022] GET /identity/v3/roles/d0854046eb544eab8092d0a773a97724 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:39.036475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:39.098914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:39.145273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a28c107-82aa-43d4-8b4a-343436083ab5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:39.179640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:39.180269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:39.180547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:39.180821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] PATH_INFO: `/v3/users/f9af3bf82a434fe98d6e0bd084529138` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:39.194301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] RBAC: Authorizing `identity:get_user(user_id=f9af3bf82a434fe98d6e0bd084529138)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:39.195732 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:39.197540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 566/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:39 2022] GET /identity/v3/users/f9af3bf82a434fe98d6e0bd084529138 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:39.203388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:39.302047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c78c1d-af9b-4af9-8e83-695a2b7c3d85 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:39.309380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:39.335902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:39.336520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:39.336801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:39.338025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:39.481610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:39.483311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:39.584115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 549/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:39.591501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:39.688127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e3b00-9c4e-4b54-ae08-e8086a84abad None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:39.769119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:39.794239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d62e9d-5137-458f-b99d-c633852bd7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:39.794897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62e9d-5137-458f-b99d-c633852bd7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:39.794897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62e9d-5137-458f-b99d-c633852bd7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:39.795475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d62e9d-5137-458f-b99d-c633852bd7d9 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:39.796623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62e9d-5137-458f-b99d-c633852bd7d9 None admin] RBAC: Authorizing `identity:get_project(project_id=8fef9c4ec38843c98a393872d0a06b56)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:39.798270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d62e9d-5137-458f-b99d-c633852bd7d9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:39.799887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 567/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:39 2022] GET /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56 => generated 311 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:39.808130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:39.821694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:39.957533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604932f5-30cc-4ca2-b554-18546fcad452 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:39.992126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:39.992732 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:39.993013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:39.993340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:39.994185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] RBAC: Adding query filter params (role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=8fef9c4ec38843c98a393872d0a06b56, user.id=f9af3bf82a434fe98d6e0bd084529138) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:39.994497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:39.996567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18885b-de82-4ad4-8406-5d1dcc6bfc95 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:40.002062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 550/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:39 2022] GET /identity/v3/role_assignments?user.id=f9af3bf82a434fe98d6e0bd084529138&scope.project.id=8fef9c4ec38843c98a393872d0a06b56&role.id=d0854046eb544eab8092d0a773a97724 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:41.886123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b61f-8820-4115-8797-7bba08cd49a2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:41.886670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b61f-8820-4115-8797-7bba08cd49a2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:41.886670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b61f-8820-4115-8797-7bba08cd49a2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:41.888140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 568/1118] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:50:41.893664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:41.894129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:41.894249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:41.913022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:41.914249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:41.980355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:41.984801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 551/1119] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:42.263136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b61f-8820-4115-8797-7bba08cd49a2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:42.363734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b61f-8820-4115-8797-7bba08cd49a2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:42.398045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:42.398666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:42.398943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:42.399215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] PATH_INFO: `/v3/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:42.400655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] RBAC: Authorizing `identity:get_role(role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:42.402054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:42.403371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 569/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:42 2022] GET /identity/v3/roles/d0854046eb544eab8092d0a773a97724 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:42.409441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:42.507414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0792e7-c767-43b2-9e67-e0cd4b50c6b5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:42.541176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:42.541830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:42.542109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:42.542382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] PATH_INFO: `/v3/users/f9af3bf82a434fe98d6e0bd084529138` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:42.543494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] RBAC: Authorizing `identity:get_user(user_id=f9af3bf82a434fe98d6e0bd084529138)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:42.544903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:42.546491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 552/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:42 2022] GET /identity/v3/users/f9af3bf82a434fe98d6e0bd084529138 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:42.552374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:42.599778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:42.653006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee32e41-a50a-4876-9a25-b83a045e74bf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:42.687458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e77a58-b450-4613-865f-347b0f5cb26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:42.688086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e77a58-b450-4613-865f-347b0f5cb26d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:42.688370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e77a58-b450-4613-865f-347b0f5cb26d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:42.688648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e77a58-b450-4613-865f-347b0f5cb26d None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:42.689777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e77a58-b450-4613-865f-347b0f5cb26d None admin] RBAC: Authorizing `identity:get_project(project_id=8fef9c4ec38843c98a393872d0a06b56)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:42.691230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e77a58-b450-4613-865f-347b0f5cb26d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:42.692850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 570/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:42 2022] GET /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:42.699799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:42.765994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:42.798958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bdd73-81c4-4064-8f58-410119a83548 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:42.833105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de035b7-d3e9-45e9-88c6-a89a926ad422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:42.833909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de035b7-d3e9-45e9-88c6-a89a926ad422 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:42.834194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de035b7-d3e9-45e9-88c6-a89a926ad422 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:42.834472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de035b7-d3e9-45e9-88c6-a89a926ad422 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56/users/f9af3bf82a434fe98d6e0bd084529138/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:42.836609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de035b7-d3e9-45e9-88c6-a89a926ad422 None admin] RBAC: Authorizing `identity:create_grant(project_id=8fef9c4ec38843c98a393872d0a06b56, user_id=f9af3bf82a434fe98d6e0bd084529138, role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:42.838432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de035b7-d3e9-45e9-88c6-a89a926ad422 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:42.939364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 553/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:42 2022] PUT /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56/users/f9af3bf82a434fe98d6e0bd084529138/roles/d0854046eb544eab8092d0a773a97724 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:50:44.833617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320dd18-6e19-48ba-9d4d-d4d89b3a66ca None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:44.834196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320dd18-6e19-48ba-9d4d-d4d89b3a66ca None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:44.834196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320dd18-6e19-48ba-9d4d-d4d89b3a66ca None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:44.835534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 571/1124] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:44.841749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:44.842168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:44.842320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:44.861907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:44.863137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:44.932153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 554/1125] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:45.199477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320dd18-6e19-48ba-9d4d-d4d89b3a66ca None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:45.298827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320dd18-6e19-48ba-9d4d-d4d89b3a66ca None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:45.313359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:45.334025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:45.334694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:45.335016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:45.335333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] PATH_INFO: `/v3/roles/d0854046eb544eab8092d0a773a97724` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:45.337059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] RBAC: Authorizing `identity:get_role(role_id=d0854046eb544eab8092d0a773a97724)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:45.338991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:45.340294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 572/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:45 2022] GET /identity/v3/roles/d0854046eb544eab8092d0a773a97724 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:45.346318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:45.404693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:45.445659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20deb346-1a87-4d3f-8b38-dd783be98f37 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:45.482710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:45.483328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:45.483565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:45.483847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] PATH_INFO: `/v3/users/f9af3bf82a434fe98d6e0bd084529138` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:45.484993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] RBAC: Authorizing `identity:get_user(user_id=f9af3bf82a434fe98d6e0bd084529138)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:45.486477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:45.488077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 555/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:45 2022] GET /identity/v3/users/f9af3bf82a434fe98d6e0bd084529138 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:45.494761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:45.593994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f2d4b-9fc9-4917-9313-f3b98ce3c45d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:45.635521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:45.636165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:45.636408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:45.636681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:45.685841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:45.704479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:45.705968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:45.804351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 573/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:45.811182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:45.912047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b74dbe-886c-4e9f-afea-01e2f145610a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:45.938231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:45.946881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5917369-99b2-487d-876f-06b9206f1605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:45.947515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5917369-99b2-487d-876f-06b9206f1605 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:45.947801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5917369-99b2-487d-876f-06b9206f1605 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:45.948081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5917369-99b2-487d-876f-06b9206f1605 None admin] PATH_INFO: `/v3/projects/8fef9c4ec38843c98a393872d0a06b56` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:45.953562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5917369-99b2-487d-876f-06b9206f1605 None admin] RBAC: Authorizing `identity:get_project(project_id=8fef9c4ec38843c98a393872d0a06b56)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:45.953562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5917369-99b2-487d-876f-06b9206f1605 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:45.953562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 556/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:45 2022] GET /identity/v3/projects/8fef9c4ec38843c98a393872d0a06b56 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:45.958551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:46.039862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:46.069461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb1c042-10e5-4c3e-83aa-4d210cc724d9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:46.092259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:46.092885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:46.093360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:46.094018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:46.094736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] RBAC: Adding query filter params (role.id=d0854046eb544eab8092d0a773a97724, scope.project.id=8fef9c4ec38843c98a393872d0a06b56, user.id=f9af3bf82a434fe98d6e0bd084529138) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:46.095046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:46.096467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24368388-deb8-4c20-a95e-723be164f230 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:46.102113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 574/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:45 2022] GET /identity/v3/role_assignments?user.id=f9af3bf82a434fe98d6e0bd084529138&scope.project.id=8fef9c4ec38843c98a393872d0a06b56&role.id=d0854046eb544eab8092d0a773a97724 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:48.040642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb75f2d-0cfe-4328-a090-1b1a470dfdb1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:48.041180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb75f2d-0cfe-4328-a090-1b1a470dfdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:48.041292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb75f2d-0cfe-4328-a090-1b1a470dfdb1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:48.042600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 557/1131] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:48.047830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:48.048266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:48.048472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:48.066660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:48.067838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:48.135894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 575/1132] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:48.409472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb75f2d-0cfe-4328-a090-1b1a470dfdb1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:48.514835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb75f2d-0cfe-4328-a090-1b1a470dfdb1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:48.549612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce340f-44fe-4d65-8b73-3bb791d0484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:48.550286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce340f-44fe-4d65-8b73-3bb791d0484c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:48.550718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce340f-44fe-4d65-8b73-3bb791d0484c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:48.550833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce340f-44fe-4d65-8b73-3bb791d0484c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:48.551980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce340f-44fe-4d65-8b73-3bb791d0484c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:48.553426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce340f-44fe-4d65-8b73-3bb791d0484c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:48.555037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 558/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:48.561925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:48.639824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:48.662174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aac69-32eb-4654-b366-a5d6f4e25b74 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:48.697299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd2368-ab3e-4bfc-939d-ad1420d20883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:48.697937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2368-ab3e-4bfc-939d-ad1420d20883 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:48.698225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2368-ab3e-4bfc-939d-ad1420d20883 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:48.698502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd2368-ab3e-4bfc-939d-ad1420d20883 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:48.699160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd2368-ab3e-4bfc-939d-ad1420d20883 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:48.700605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd2368-ab3e-4bfc-939d-ad1420d20883 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:48.840095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 576/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:50:48 2022] POST /identity/v3/projects => generated 311 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:50:50.789477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5e0c1-d442-4de6-8bba-63cc107ba116 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:50.790090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5e0c1-d442-4de6-8bba-63cc107ba116 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:50.790283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b5e0c1-d442-4de6-8bba-63cc107ba116 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:50.791813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 559/1135] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:50.796992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:50.797442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:50.797543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:50.816931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:50.818162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:50.886707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 577/1136] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:51.159738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5e0c1-d442-4de6-8bba-63cc107ba116 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:51.226683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:51.266721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b5e0c1-d442-4de6-8bba-63cc107ba116 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:51.300329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945456d0-0d59-4149-8527-6b62022872d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:51.300952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945456d0-0d59-4149-8527-6b62022872d1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:51.301235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945456d0-0d59-4149-8527-6b62022872d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:51.301512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945456d0-0d59-4149-8527-6b62022872d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:51.302695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945456d0-0d59-4149-8527-6b62022872d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:51.304121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945456d0-0d59-4149-8527-6b62022872d1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:51.306303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 560/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:50:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:51.312110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:51.371712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:51.413286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d91c0c-618f-4125-80ee-f6d39a3b7d3a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:51.448247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032aa97-e1b2-410b-8c87-9a66711de3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:51.448853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032aa97-e1b2-410b-8c87-9a66711de3c4 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:51.449130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032aa97-e1b2-410b-8c87-9a66711de3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:51.449408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032aa97-e1b2-410b-8c87-9a66711de3c4 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:51.450081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032aa97-e1b2-410b-8c87-9a66711de3c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:51.451511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032aa97-e1b2-410b-8c87-9a66711de3c4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:51.593717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 578/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:50:51 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:50:53.565627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e5738-c16d-43fc-acfd-4c97a3f4bf14 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:53.566230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e5738-c16d-43fc-acfd-4c97a3f4bf14 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:53.566230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e5738-c16d-43fc-acfd-4c97a3f4bf14 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:53.567555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 561/1139] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:53.573093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:53.573513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:53.573647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:53.592104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:53.593325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:53.661557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 579/1140] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:53.955400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e5738-c16d-43fc-acfd-4c97a3f4bf14 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:54.060208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e5738-c16d-43fc-acfd-4c97a3f4bf14 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:54.096884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:54.103085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:54.103722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:54.104010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:54.104290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:54.114632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:54.119556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:54.120777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:54.121950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 562/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:54.127016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:54.173076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:54.228352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ad92c5-736c-460a-af21-1f4355b59feb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:54.262213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:54.262818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:54.263105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:54.263381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:54.264039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:54.264352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:54.265717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:54.274111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 580/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:54.280457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:54.379940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ecf6e-bb5a-4979-a435-28f21322b69e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:54.424640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:54.427086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:54.427393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:54.427712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] PATH_INFO: `/v3/users/11006d2b60db4d958931797d0782ae7b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:54.444588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] RBAC: Authorizing `identity:get_user(user_id=11006d2b60db4d958931797d0782ae7b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:54.446198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:54.448942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 563/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:54 2022] GET /identity/v3/users/11006d2b60db4d958931797d0782ae7b => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:54.455025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:54.557493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bb8d9-65d5-483b-8414-3a4b60589fe7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:54.564511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:54.591257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:54.591881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:54.592140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:54.592417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:54.658487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:54.659948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:54.759123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 581/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:50:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:50:54.765399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:54.814422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:54.865269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f49439-feac-40f3-8580-04085bc7c332 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:54.899250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97119ceb-3627-42bf-beec-466948d85236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:54.900378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119ceb-3627-42bf-beec-466948d85236 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:54.900661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119ceb-3627-42bf-beec-466948d85236 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:54.900941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97119ceb-3627-42bf-beec-466948d85236 None admin] PATH_INFO: `/v3/projects/9ef0ad80bdc2498386267cc8b85105c9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:54.902104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119ceb-3627-42bf-beec-466948d85236 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef0ad80bdc2498386267cc8b85105c9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:54.903545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97119ceb-3627-42bf-beec-466948d85236 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:54.905178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 564/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:54 2022] GET /identity/v3/projects/9ef0ad80bdc2498386267cc8b85105c9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:54.911405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:54.919508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:55.011858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c9920-f513-4299-9dae-e3ac29eb2dbe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:55.046900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:55.047502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:55.047778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:55.048047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:55.048761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=9ef0ad80bdc2498386267cc8b85105c9, user.id=11006d2b60db4d958931797d0782ae7b) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:55.049069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:55.050525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd574530-fcbd-4f8f-af86-051cb9f4b5e8 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:55.055830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 582/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:50:54 2022] GET /identity/v3/role_assignments?user.id=11006d2b60db4d958931797d0782ae7b&scope.project.id=9ef0ad80bdc2498386267cc8b85105c9&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:56.995242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87c1a-d00e-4c39-a4f5-42fcffdb2be4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:56.995808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87c1a-d00e-4c39-a4f5-42fcffdb2be4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:56.995808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d87c1a-d00e-4c39-a4f5-42fcffdb2be4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:56.997141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 565/1147] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:50:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:50:57.002431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:57.002821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:57.002989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:57.021639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:50:57.022873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:50:57.086166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:57.091514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 583/1148] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:50:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:50:57.373316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87c1a-d00e-4c39-a4f5-42fcffdb2be4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:57.474956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d87c1a-d00e-4c39-a4f5-42fcffdb2be4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:57.492299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:57.509831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:57.510685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:57.510798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:57.511029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:57.521016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:50:57.526336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:50:57.527441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:50:57.528483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 566/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:50:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:50:57.533542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:57.635350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0524256-4e88-4b98-8956-dac18d5e8b8c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:57.669478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:57.670113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:57.670398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:57.670675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:57.671330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:50:57.671635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:57.673007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:57.681119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 584/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:50:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:57.687361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:57.786394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17729d4e-654e-4b1b-8ca0-9e226af39b11 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:57.819474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:57.820097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:57.820378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:57.820653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] PATH_INFO: `/v3/users/11006d2b60db4d958931797d0782ae7b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:57.822314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] RBAC: Authorizing `identity:get_user(user_id=11006d2b60db4d958931797d0782ae7b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:57.823727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:57.825306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 567/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:50:57 2022] GET /identity/v3/users/11006d2b60db4d958931797d0782ae7b => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:57.831178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:57.903396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:57.941876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8b2968-2508-4b86-9105-f8e8923a7014 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:57.976217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe675f91-6c8f-4d64-8152-9ad1d7624187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:57.976842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675f91-6c8f-4d64-8152-9ad1d7624187 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:57.977167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675f91-6c8f-4d64-8152-9ad1d7624187 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:57.977629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe675f91-6c8f-4d64-8152-9ad1d7624187 None admin] PATH_INFO: `/v3/projects/9ef0ad80bdc2498386267cc8b85105c9` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:57.978799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675f91-6c8f-4d64-8152-9ad1d7624187 None admin] RBAC: Authorizing `identity:get_project(project_id=9ef0ad80bdc2498386267cc8b85105c9)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:57.980258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe675f91-6c8f-4d64-8152-9ad1d7624187 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:57.981940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 585/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:50:57 2022] GET /identity/v3/projects/9ef0ad80bdc2498386267cc8b85105c9 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:50:57.988147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:50:58.045437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:50:58.099442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e8ffe-6f9c-48c2-bc7b-672d8e35b5f6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:50:58.136846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd24bb-5c71-4908-ae8a-29b80cffc0fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:50:58.137511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd24bb-5c71-4908-ae8a-29b80cffc0fa None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:50:58.137824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd24bb-5c71-4908-ae8a-29b80cffc0fa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:50:58.138107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd24bb-5c71-4908-ae8a-29b80cffc0fa None admin] PATH_INFO: `/v3/projects/9ef0ad80bdc2498386267cc8b85105c9/users/11006d2b60db4d958931797d0782ae7b/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:50:58.140023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd24bb-5c71-4908-ae8a-29b80cffc0fa None admin] RBAC: Authorizing `identity:create_grant(project_id=9ef0ad80bdc2498386267cc8b85105c9, user_id=11006d2b60db4d958931797d0782ae7b, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:50:58.141452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd24bb-5c71-4908-ae8a-29b80cffc0fa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:50:58.253133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 568/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:50:57 2022] PUT /identity/v3/projects/9ef0ad80bdc2498386267cc8b85105c9/users/11006d2b60db4d958931797d0782ae7b/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:51:00.187264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ffb9-271d-4793-a6f8-a9cc7313576b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:00.187819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ffb9-271d-4793-a6f8-a9cc7313576b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:00.187819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce75ffb9-271d-4793-a6f8-a9cc7313576b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:00.189141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 586/1154] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:00.194279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:00.194669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:00.194846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:00.214243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:00.215555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:00.282441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 569/1155] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:00.557316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ffb9-271d-4793-a6f8-a9cc7313576b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:00.658659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce75ffb9-271d-4793-a6f8-a9cc7313576b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:00.687046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:00.692628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:00.693263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:00.693550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:00.693857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:00.706108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:00.711059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:00.713214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:51:00.714128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 587/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:00.719149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:00.753809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:00.822212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8bcdf-b6de-4691-9186-ac33cd8ee7aa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:00.857456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:00.858107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:00.858393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:00.858671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:00.859359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:00.859667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:00.861031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:00.869513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 570/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:00.875632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:00.977422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf3547-71aa-47fd-9fa5-fce29015dfbd None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:01.011191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:01.011749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:01.012034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:01.012311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] PATH_INFO: `/v3/users/11006d2b60db4d958931797d0782ae7b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:01.013430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] RBAC: Authorizing `identity:get_user(user_id=11006d2b60db4d958931797d0782ae7b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:01.014925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:01.016503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 588/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:51:00 2022] GET /identity/v3/users/11006d2b60db4d958931797d0782ae7b => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:01.022486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:01.117104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:01.123497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afe431-36d1-42ca-ab41-75da0442b96c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:01.157431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:01.158069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:01.158358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:01.158643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:01.225315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:01.226808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:01.322461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:01.328998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 571/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:51:01.334945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:01.392948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:01.434673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbf7ab-6ef7-4ae3-965d-979a32ebebd5 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:01.469527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5864e0-00f1-4847-8e4c-98ad07597c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:01.470182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5864e0-00f1-4847-8e4c-98ad07597c0f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:01.470463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5864e0-00f1-4847-8e4c-98ad07597c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:01.470734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5864e0-00f1-4847-8e4c-98ad07597c0f None admin] PATH_INFO: `/v3/projects/9ef0ad80bdc2498386267cc8b85105c9` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:01.471837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5864e0-00f1-4847-8e4c-98ad07597c0f None admin] RBAC: Authorizing `identity:get_project(project_id=9ef0ad80bdc2498386267cc8b85105c9)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:01.473267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5864e0-00f1-4847-8e4c-98ad07597c0f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:01.474913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 589/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:51:01 2022] GET /identity/v3/projects/9ef0ad80bdc2498386267cc8b85105c9 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:01.481607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:01.580576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2504ea-1a69-48de-b627-7ecefeeb1d21 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:01.614173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:01.615051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:01.615521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:01.615807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:01.616532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=9ef0ad80bdc2498386267cc8b85105c9, user.id=11006d2b60db4d958931797d0782ae7b) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:01.616847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:01.619532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab22166-d6e4-4154-b806-30041537511a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:01.625459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 572/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:01 2022] GET /identity/v3/role_assignments?user.id=11006d2b60db4d958931797d0782ae7b&scope.project.id=9ef0ad80bdc2498386267cc8b85105c9&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:03.582028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b1972-9f06-4e61-90c1-b2d85cda8ef0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:03.582585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b1972-9f06-4e61-90c1-b2d85cda8ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:03.582585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b1972-9f06-4e61-90c1-b2d85cda8ef0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:03.583887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 590/1162] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:03.589470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:03.589918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:03.590069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:03.608155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:03.609348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:03.667685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:03.678673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 573/1163] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:03.947154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b1972-9f06-4e61-90c1-b2d85cda8ef0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:04.049453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b1972-9f06-4e61-90c1-b2d85cda8ef0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:04.075099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:04.083656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:04.085444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:04.085444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:04.085444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:04.153458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:04.154945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:04.255013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 591/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:51:04.260898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:04.360959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d1665-ab10-471b-aadf-90f81a7cb16f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:04.394212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:04.394850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:04.395132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:04.395405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:04.402919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:04.404353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3a1b781c-1756-4149-a4d5-dfd842b745a9 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:04.416284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:04.417548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:51:04.418494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 574/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:51:04 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:04.423501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:04.449661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:04.523469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981aa5-42dd-4aa1-90dd-dca4f41571bf None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:04.558044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:04.558651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:04.558939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:04.559218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:04.559855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:04.560165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:04.561540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1a81a3-14da-4af2-b3d3-cb32e8f38fd9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:04.568692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 592/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Tue Aug 30 00:51:04 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:06.465578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030bb10-5951-40f3-b394-1f6b06c15bf5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:06.466138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030bb10-5951-40f3-b394-1f6b06c15bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:06.466229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030bb10-5951-40f3-b394-1f6b06c15bf5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:06.467525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 575/1167] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:06.472944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ed174-9bc8-41ca-92ff-67375d956ff0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:06.473379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ed174-9bc8-41ca-92ff-67375d956ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:06.473508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ed174-9bc8-41ca-92ff-67375d956ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:06.492602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-df8ed174-9bc8-41ca-92ff-67375d956ff0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:06.493924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-df8ed174-9bc8-41ca-92ff-67375d956ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:06.560745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 593/1168] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:06.830343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030bb10-5951-40f3-b394-1f6b06c15bf5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:06.882735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:06.934606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030bb10-5951-40f3-b394-1f6b06c15bf5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:06.968434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235629d-6f1b-49ad-a5db-5f836f12c65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:06.969045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235629d-6f1b-49ad-a5db-5f836f12c65d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:06.969330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235629d-6f1b-49ad-a5db-5f836f12c65d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:06.969610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235629d-6f1b-49ad-a5db-5f836f12c65d None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:06.970310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235629d-6f1b-49ad-a5db-5f836f12c65d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:06.971676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235629d-6f1b-49ad-a5db-5f836f12c65d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:07.059760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 576/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Tue Aug 30 00:51:06 2022] POST /identity/v3/domains => generated 267 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:51:08.989648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bc861-63ee-4b6c-b140-41355a4ff3e4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:08.990209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bc861-63ee-4b6c-b140-41355a4ff3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:08.990209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bc861-63ee-4b6c-b140-41355a4ff3e4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:08.991572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 594/1170] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:08.996838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:08.997258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:08.997407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:09.016370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:09.017583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:09.092071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 577/1171] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:09.356263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bc861-63ee-4b6c-b140-41355a4ff3e4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:09.384012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:09.466886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bc861-63ee-4b6c-b140-41355a4ff3e4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:09.501080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e761e2-6eeb-40e5-9e76-6c1d25a9700c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:09.501722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e761e2-6eeb-40e5-9e76-6c1d25a9700c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:09.502032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e761e2-6eeb-40e5-9e76-6c1d25a9700c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:09.502311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e761e2-6eeb-40e5-9e76-6c1d25a9700c None admin] PATH_INFO: `/v3/domains/4dd153e15e74482e97c5bdf676caf2ca` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:09.512675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e761e2-6eeb-40e5-9e76-6c1d25a9700c None admin] RBAC: Authorizing `identity:get_domain(domain_id=4dd153e15e74482e97c5bdf676caf2ca)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:09.514177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e761e2-6eeb-40e5-9e76-6c1d25a9700c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:09.515786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 595/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Aug 30 00:51:09 2022] GET /identity/v3/domains/4dd153e15e74482e97c5bdf676caf2ca => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:09.521757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:09.562829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:09.620052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd04eb-2287-40a4-8910-4bc4aea3ba54 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:09.654103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc8e7b-f283-4819-a54b-0939538ce3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:09.654711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc8e7b-f283-4819-a54b-0939538ce3ff None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:09.654994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc8e7b-f283-4819-a54b-0939538ce3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:09.655272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc8e7b-f283-4819-a54b-0939538ce3ff None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:09.655917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc8e7b-f283-4819-a54b-0939538ce3ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:09.657365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dc8e7b-f283-4819-a54b-0939538ce3ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:09.742633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 578/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:51:09 2022] POST /identity/v3/projects => generated 361 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:51:12.010870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab591c68-8175-4907-ad34-f773401c0e53 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:12.011400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab591c68-8175-4907-ad34-f773401c0e53 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:12.011400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab591c68-8175-4907-ad34-f773401c0e53 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:12.012752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 596/1174] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:12.018137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:12.018543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:12.018702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:12.037106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:12.038341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:12.105410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 579/1175] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:12.377781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab591c68-8175-4907-ad34-f773401c0e53 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:12.463327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:12.484273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab591c68-8175-4907-ad34-f773401c0e53 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:12.519723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e1cd4-dfcf-4f76-b86a-ad5a790f66a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:12.520354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e1cd4-dfcf-4f76-b86a-ad5a790f66a4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:12.520637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e1cd4-dfcf-4f76-b86a-ad5a790f66a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:12.520913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e1cd4-dfcf-4f76-b86a-ad5a790f66a4 None admin] PATH_INFO: `/v3/domains/4dd153e15e74482e97c5bdf676caf2ca` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:12.522060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e1cd4-dfcf-4f76-b86a-ad5a790f66a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=4dd153e15e74482e97c5bdf676caf2ca)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:12.524024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e1cd4-dfcf-4f76-b86a-ad5a790f66a4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:12.525615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 597/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Tue Aug 30 00:51:12 2022] GET /identity/v3/domains/4dd153e15e74482e97c5bdf676caf2ca => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:12.531581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:12.557203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:12.632384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224693c4-231e-4c0c-ad6a-f011a3e1339c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:12.667798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19624209-4c81-4b7e-8a2c-7f689b2ef315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:12.668396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624209-4c81-4b7e-8a2c-7f689b2ef315 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:12.668675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624209-4c81-4b7e-8a2c-7f689b2ef315 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:12.668948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19624209-4c81-4b7e-8a2c-7f689b2ef315 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:12.669607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624209-4c81-4b7e-8a2c-7f689b2ef315 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:12.671058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19624209-4c81-4b7e-8a2c-7f689b2ef315 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:12.781651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 580/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:51:12 2022] POST /identity/v3/users => generated 311 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 00:51:14.780766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497ebc2-6efd-4cb4-bd94-6f498fee4b29 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:14.781290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497ebc2-6efd-4cb4-bd94-6f498fee4b29 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:14.781290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497ebc2-6efd-4cb4-bd94-6f498fee4b29 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:14.782619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 598/1178] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:14.788000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:14.788410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:14.788560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:14.812585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:14.813817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:14.882410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 581/1179] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:15.155231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497ebc2-6efd-4cb4-bd94-6f498fee4b29 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:15.259220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497ebc2-6efd-4cb4-bd94-6f498fee4b29 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:15.292743 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:15.293365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:15.293649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:15.293951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:15.303999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:15.309114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:15.310258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:51:15.311488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 599/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:15.316343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:15.327346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:15.417125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc765c7-005c-4cab-8f7d-61ea79e9d1c1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:15.453137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:15.453808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:15.454061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:15.454342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:15.455012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:15.455352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:15.456687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:15.468505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 582/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:15.474502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:15.486318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:15.658810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed9382-03b2-44c6-b2b3-ccc86ba41478 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:15.692565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:15.693119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:15.693402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:15.693679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] PATH_INFO: `/v3/users/924da228ab574b48a219a26405cfe735` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:15.707636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] RBAC: Authorizing `identity:get_user(user_id=924da228ab574b48a219a26405cfe735)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:15.709089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:15.711001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 600/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:51:15 2022] GET /identity/v3/users/924da228ab574b48a219a26405cfe735 => generated 311 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:15.717209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:15.787215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:15.818299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2200e2-0f6e-4dd8-95e9-11e12a7f882f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:15.852934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:15.853540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:15.853852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:15.854129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:15.919756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:15.921194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:15.986546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:16.016880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 583/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:51:16.023140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:16.089923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:16.124161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee2d4c-fb8b-4062-adb6-523e6b942d6c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:16.158000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15d202-d5e1-41a0-8abd-3d96016e099c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:16.158627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d202-d5e1-41a0-8abd-3d96016e099c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:16.158905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d202-d5e1-41a0-8abd-3d96016e099c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:16.159177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15d202-d5e1-41a0-8abd-3d96016e099c None admin] PATH_INFO: `/v3/projects/0216e7351cf045f48577f67fbef56d19` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:16.160272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15d202-d5e1-41a0-8abd-3d96016e099c None admin] RBAC: Authorizing `identity:get_project(project_id=0216e7351cf045f48577f67fbef56d19)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:16.161712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15d202-d5e1-41a0-8abd-3d96016e099c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:16.163358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 601/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:51:16 2022] GET /identity/v3/projects/0216e7351cf045f48577f67fbef56d19 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:16.170385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:16.273181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96640a42-0865-475a-ae8d-9c875f6f7628 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:16.308283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:16.308886 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:16.309165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:16.309441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:16.310186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=0216e7351cf045f48577f67fbef56d19, user.id=924da228ab574b48a219a26405cfe735) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:16.310500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:16.311915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851f214-b24f-4ea2-aa7b-c5fb1711c1df None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:16.317231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 584/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:16 2022] GET /identity/v3/role_assignments?user.id=924da228ab574b48a219a26405cfe735&scope.project.id=0216e7351cf045f48577f67fbef56d19&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:18.227229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d9795-7b25-40fb-9fa0-57721a2787dd None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:18.227784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d9795-7b25-40fb-9fa0-57721a2787dd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:18.227931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d9795-7b25-40fb-9fa0-57721a2787dd None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:18.229221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 602/1186] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:18.234882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:18.235348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:18.235438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:18.254948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:18.256142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:18.283992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:18.324893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 585/1187] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 00:51:18.593918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d9795-7b25-40fb-9fa0-57721a2787dd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:18.697164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d9795-7b25-40fb-9fa0-57721a2787dd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:18.731427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:18.732051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:18.732334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:18.732609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:18.734493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:18.748829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:18.749865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:51:18.750988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 603/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:18.755761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:18.882906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012745bb-4056-4406-b05a-95f13f23f46d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:18.917296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:18.917931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:18.918213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:18.918488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:18.919141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:18.919444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:18.920797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:18.928608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 586/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:18.935295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:19.032637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19d9f9-9baa-48fb-9b88-cae7f1280ad0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:19.068188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:19.068828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:19.069110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:19.069387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] PATH_INFO: `/v3/users/924da228ab574b48a219a26405cfe735` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:19.070743 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] RBAC: Authorizing `identity:get_user(user_id=924da228ab574b48a219a26405cfe735)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:19.072182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:19.073789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 604/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:51:18 2022] GET /identity/v3/users/924da228ab574b48a219a26405cfe735 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:19.079790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:19.108712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:19.184920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc3c63-55ad-41b9-93f6-aed2673785c7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:19.220008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6b9cc-bbb7-4e18-96e0-7dc4d145a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:19.220645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6b9cc-bbb7-4e18-96e0-7dc4d145a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:19.220930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6b9cc-bbb7-4e18-96e0-7dc4d145a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:19.221208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6b9cc-bbb7-4e18-96e0-7dc4d145a9a9 None admin] PATH_INFO: `/v3/projects/0216e7351cf045f48577f67fbef56d19` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:19.222349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6b9cc-bbb7-4e18-96e0-7dc4d145a9a9 None admin] RBAC: Authorizing `identity:get_project(project_id=0216e7351cf045f48577f67fbef56d19)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:19.223791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6b9cc-bbb7-4e18-96e0-7dc4d145a9a9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:19.225495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 587/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:51:19 2022] GET /identity/v3/projects/0216e7351cf045f48577f67fbef56d19 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:19.232503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:19.294336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:19.331472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7901a338-0d4c-4db7-aec2-1fd06fe38438 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:19.366213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7d691-02c7-4eee-8a44-03fec4a2c9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:19.366970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d691-02c7-4eee-8a44-03fec4a2c9aa None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:19.367319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d691-02c7-4eee-8a44-03fec4a2c9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:19.367598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7d691-02c7-4eee-8a44-03fec4a2c9aa None admin] PATH_INFO: `/v3/projects/0216e7351cf045f48577f67fbef56d19/users/924da228ab574b48a219a26405cfe735/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:19.369491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d691-02c7-4eee-8a44-03fec4a2c9aa None admin] RBAC: Authorizing `identity:create_grant(project_id=0216e7351cf045f48577f67fbef56d19, user_id=924da228ab574b48a219a26405cfe735, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:19.370974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec7d691-02c7-4eee-8a44-03fec4a2c9aa None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:19.434851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 605/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:51:19 2022] PUT /identity/v3/projects/0216e7351cf045f48577f67fbef56d19/users/924da228ab574b48a219a26405cfe735/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:51:21.341939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77edcca-b0e0-44cf-8851-44a0472b1538 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:21.342498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77edcca-b0e0-44cf-8851-44a0472b1538 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:21.342498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77edcca-b0e0-44cf-8851-44a0472b1538 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:21.343849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 588/1193] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:51:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:51:21.349128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:21.349516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:21.349684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:21.368736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:51:21.369967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:51:21.437399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 606/1194] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:51:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:51:21.717013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77edcca-b0e0-44cf-8851-44a0472b1538 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:21.810624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:21.818131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77edcca-b0e0-44cf-8851-44a0472b1538 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:21.853046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:21.853674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:21.853978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:21.854249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:21.864222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:51:21.868979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:51:21.870118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:51:21.871341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 589/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:51:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:51:21.876198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:21.928743 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:21.976600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515fe45e-b2d6-4470-9b75-8bd753299712 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:22.012046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:22.012657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:22.012956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:22.013233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:22.013923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:22.014234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:22.015603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:22.024306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 607/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:51:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:22.030150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:22.129219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009a39f-d054-49c4-bca0-fe2147946af6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:22.162370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:22.162993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:22.163278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:22.163557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] PATH_INFO: `/v3/users/924da228ab574b48a219a26405cfe735` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:22.164781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] RBAC: Authorizing `identity:get_user(user_id=924da228ab574b48a219a26405cfe735)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:22.166233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:22.167805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 590/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Tue Aug 30 00:51:22 2022] GET /identity/v3/users/924da228ab574b48a219a26405cfe735 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:22.173800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:22.273506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bb8fc-d1ca-4952-bc40-ed9cec1d9198 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:22.286622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:22.306708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:22.307358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:22.307641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:22.307919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:22.375063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:22.376515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:22.482037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 608/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:51:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:51:22.488122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:22.513548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:22.590074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dfb36c-2dad-4835-bba2-2951ce1909b7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:22.623287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21156171-df34-4d6d-9188-1f4842868a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:22.623917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21156171-df34-4d6d-9188-1f4842868a11 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:22.624199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21156171-df34-4d6d-9188-1f4842868a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:22.624476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21156171-df34-4d6d-9188-1f4842868a11 None admin] PATH_INFO: `/v3/projects/0216e7351cf045f48577f67fbef56d19` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:22.625591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21156171-df34-4d6d-9188-1f4842868a11 None admin] RBAC: Authorizing `identity:get_project(project_id=0216e7351cf045f48577f67fbef56d19)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:22.627048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21156171-df34-4d6d-9188-1f4842868a11 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:22.628670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 591/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Tue Aug 30 00:51:22 2022] GET /identity/v3/projects/0216e7351cf045f48577f67fbef56d19 => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:22.635454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:51:22.650981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:22.733208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7706e-8281-4470-81eb-dbcf8b271888 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:51:22.766348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:51:22.766942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:51:22.767223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:51:22.767494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:51:22.768203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=0216e7351cf045f48577f67fbef56d19, user.id=924da228ab574b48a219a26405cfe735) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:51:22.768508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:51:22.769957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af4018e-aec1-428f-b58d-2a87d0940497 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:51:22.776270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 609/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:51:22 2022] GET /identity/v3/role_assignments?user.id=924da228ab574b48a219a26405cfe735&scope.project.id=0216e7351cf045f48577f67fbef56d19&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:51:32.618981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=55 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:51:32.773752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=61 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:49.922893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5934-461b-4164-974d-6b12593b5616 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:49.923492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5934-461b-4164-974d-6b12593b5616 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:49.923492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ad5934-461b-4164-974d-6b12593b5616 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:49.924773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 592/1201] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:49.929930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:49.930393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:49.930486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:49.948746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:49.949993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:50.016834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 610/1202] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:52:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:52:50.292255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5934-461b-4164-974d-6b12593b5616 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:50.400864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ad5934-461b-4164-974d-6b12593b5616 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:50.437399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:50.438063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:50.438347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:50.438624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:50.449656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:50.451140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:52:50.466628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:50.468189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:52:50.469508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:52:50.469508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:50.469508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:52:50.469508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 593/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:50.474077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:50.579220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533c456c-3b27-4839-8c0e-2a11f337f218 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:50.614724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:50.622004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:50.622648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:50.623020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:50.623309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:50.623957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:50.624266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:50.625645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435b306-0ab0-4c8f-ac42-3fc76125f1ab None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:50.638487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 611/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:52:50.644554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:50.705738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:50.752668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aae6be-38aa-4369-b597-2538df1ea07f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:50.786217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afca71-bd01-479a-a4cf-1646926bb612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:50.786822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afca71-bd01-479a-a4cf-1646926bb612 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:50.787106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afca71-bd01-479a-a4cf-1646926bb612 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:50.787385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afca71-bd01-479a-a4cf-1646926bb612 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:50.788026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afca71-bd01-479a-a4cf-1646926bb612 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:50.789453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afca71-bd01-479a-a4cf-1646926bb612 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:51.027359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 594/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Tue Aug 30 00:52:50 2022] POST /identity/v3/users => generated 251 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:52:53.011028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a9b61-7b2d-4297-9e76-d028cfb32ae6 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:53.011574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a9b61-7b2d-4297-9e76-d028cfb32ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:53.011574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a9b61-7b2d-4297-9e76-d028cfb32ae6 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:53.013007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 612/1206] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 00:52:53.018593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:53.018988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:53.019151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:53.039662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:53.040861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:53.109594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 595/1207] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:52:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:52:53.391007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a9b61-7b2d-4297-9e76-d028cfb32ae6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:53.496217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a9b61-7b2d-4297-9e76-d028cfb32ae6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:53.530683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:53.531302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:53.531585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:53.531898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:53.542289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:52:53.546980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:53.548087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:53.549364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 613/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:52:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:52:53.554407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:53.593868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:53.653796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a072e8-8c12-4023-8d9a-92c8a889bc4b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:53.687142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:53.687751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:53.688034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:53.688312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:53.688972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:53.689279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:53.690671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:53.700270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 596/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:52:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:52:53.706372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:53.759119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:53.809945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc203c-9687-49cd-9469-05e0f3c07ea8 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:53.843599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:53.844220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:53.844499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:53.844771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:53.856023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:53.857469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:52:53.871744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:53.872876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:52:53.873856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:52:53.873856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:53.873856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:53.873856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 614/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:53.878851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:53.981558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:53.984463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319dc270-bf31-4fc3-a5e1-ccbd3a87c76f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:54.018948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:54.020265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:54.020567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:54.020843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:54.021483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:54.021814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:54.023173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:54.034848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 597/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:52:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:52:54.040812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:54.145934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd504a14-afb8-4e64-9478-2e929eafd794 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:54.147464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:54.179529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:54.180158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:54.180438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:54.180709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:54.188201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:54.189718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:52:54.201484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:54.208870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:54.209888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 615/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:54.210564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:54.310077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f0a6f-ab03-4f23-8fb1-beadd024f902 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:54.346281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:54.346890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:54.347167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:54.347573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:54.348547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:54.348853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:54.350369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:54.368841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 598/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 30 00:52:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:54.374961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:54.482502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a0dd27-70ac-48fe-90f3-1272e512eb6b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:54.518638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:54.519288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:54.519575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:54.519939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:54.554432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:54.597447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:54.598906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:54.699222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 616/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:52:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:52:54.705271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:54.726347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:54.808779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb1059-140f-46d7-b50f-35e329f4f1c1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:54.843239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:54.846741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:54.846741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:54.846741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:54.855242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:54.856624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:52:54.871962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:54.873210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:52:54.874356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:52:54.874356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:54.874356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:52:54.874356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 599/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:54.879012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:54.942276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:54.990964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4087689-18ba-43a2-a472-4afd03657f42 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:55.024489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:55.025101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:55.025383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:55.025658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:55.026342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:55.026646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:55.028040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:55.040548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 617/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:55.046454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:55.138839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:55.174446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36da0e0e-c4fc-4529-be31-bd1bf76d10d2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:55.209412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:55.210080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:55.210370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:55.210650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:55.217588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:55.219083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:52:55.230799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:55.231920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:52:55.233030 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 600/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:55.238732 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:55.338845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df5da2-9394-44c9-8e8c-c7ddbdea51ae None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:55.360638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:55.372189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:55.372788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:55.373066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:55.373340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:55.374058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:55.374357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:55.375782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc7601-9418-443c-a6b9-88806bb28de1 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:55.387367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 618/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:55.393833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:55.499699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15412d-cece-4f47-b2a7-54449c834448 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:55.528502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:55.534504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:55.535133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:55.535412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:55.535687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:55.536401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=38d22a2e2f374aa6bba35838ebde3b4c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:55.536708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:55.538179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47850a88-4485-41fd-ad1d-e22d123579b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:55.543458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 601/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:52:55 2022] GET /identity/v3/role_assignments?user.id=38d22a2e2f374aa6bba35838ebde3b4c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:57.464134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0325f9-f935-4aed-aa95-47cb1a39c8bb None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:57.464721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0325f9-f935-4aed-aa95-47cb1a39c8bb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:57.464721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0325f9-f935-4aed-aa95-47cb1a39c8bb None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:57.466063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 619/1220] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:52:57.471363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:57.471800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:57.471950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:57.504325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:52:57.504325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:52:57.562980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 602/1221] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:52:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:52:57.829138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0325f9-f935-4aed-aa95-47cb1a39c8bb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:57.932873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0325f9-f935-4aed-aa95-47cb1a39c8bb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:57.967172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:57.967799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:57.968083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:57.968356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:57.979378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:52:57.984203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:57.988559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:57.990076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 620/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:52:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:57.992143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:58.095848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7eb6e0-d311-41ee-8fb3-40df3a02ac31 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:58.121588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:58.132417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:58.133056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:58.133338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:58.133619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:58.134311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:58.134616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:58.136000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:58.144883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 603/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:52:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:58.153368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:58.196333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:58.253227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2bb68-d5a0-4fb1-81f8-a929c09d2be6 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:58.287144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:58.290983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:58.290983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:58.290983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:58.298377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:58.299823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:52:58.315266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:58.320665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:52:58.321793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:52:58.321793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:58.321793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:58.321793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 621/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:58.322636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:58.426599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700aeb7d-5af8-4d85-8486-acd9a62a94ee None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:58.462860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:58.463938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:58.464054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:58.464302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:58.465413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:58.465748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:58.467156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:58.480836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 604/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:52:58.486981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:58.584077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:58.591302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab77099-1592-4b17-b8cc-e0124f81f031 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:58.626043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:58.626751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:58.627029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:58.627298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:58.636509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:58.638006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:52:58.650357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:58.651756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:58.653103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 622/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:58.657467 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:58.704324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:58.756094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213b0ea-185e-4373-afb4-5408a1b12aa8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:58.790342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:58.790941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:58.791217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:58.791489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:58.792203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:58.792507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:58.793949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:58.811995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 605/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 30 00:52:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:58.818711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:58.927120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f24e6-cb0e-4133-a694-1844fa95b9e7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:58.965937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:58.965937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:58.965937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:58.965937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:58.974174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:58.975635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:52:58.990507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:58.996639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:52:58.997823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:52:58.997823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:52:58.997823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:58.997823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 623/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:58.999179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:59.109905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ccc04d-3c8b-4a62-bef3-0735023764f4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:59.112227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:59.144773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:59.145382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:59.145666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:59.145976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:59.146621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:59.146928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:59.148421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:59.159717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 606/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:52:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:59.166411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:59.176380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:59.271276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822e601-6e0e-4579-8c7b-a3ecb9616b9b None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:59.304083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:59.309569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:59.309569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:59.309569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:59.311924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:59.313370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:52:59.325448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:52:59.326565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:52:59.327479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 624/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:52:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:52:59.333268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:59.434915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e69f6-9b6a-4514-919d-b5a7865ec929 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:59.468468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:59.469124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:59.469408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:59.469686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:59.470410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:52:59.470717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:59.472746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14932f21-21a4-4e62-9dc9-2b2484e464be None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:59.485205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 607/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:52:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:52:59.492335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:52:59.560282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:52:59.596289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b8eb-005a-4aa2-93b9-7f297f62d5a2 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:52:59.630716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cda4e1-0736-4e8d-9847-217cfa3d41e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:52:59.631383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cda4e1-0736-4e8d-9847-217cfa3d41e9 None admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:52:59.631664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cda4e1-0736-4e8d-9847-217cfa3d41e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:52:59.631941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cda4e1-0736-4e8d-9847-217cfa3d41e9 None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/38d22a2e2f374aa6bba35838ebde3b4c/roles/89e918e9fe3a440498130d9fe2d5a464` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:52:59.647300 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cda4e1-0736-4e8d-9847-217cfa3d41e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=38d22a2e2f374aa6bba35838ebde3b4c, role_id=89e918e9fe3a440498130d9fe2d5a464)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:52:59.651781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cda4e1-0736-4e8d-9847-217cfa3d41e9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:52:59.745636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 625/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:52:59 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/38d22a2e2f374aa6bba35838ebde3b4c/roles/89e918e9fe3a440498130d9fe2d5a464 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 00:53:01.744773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497f7a4-c7bf-4099-81cd-f60d9c28778d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:01.745289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497f7a4-c7bf-4099-81cd-f60d9c28778d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:01.745289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497f7a4-c7bf-4099-81cd-f60d9c28778d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:01.746590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 608/1233] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:01.751788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:01.752176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:01.752350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:01.771712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:01.774197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:01.842410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 626/1234] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:02.114226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497f7a4-c7bf-4099-81cd-f60d9c28778d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:02.140074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:02.220109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497f7a4-c7bf-4099-81cd-f60d9c28778d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:02.255272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:02.255895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:02.256749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:02.256749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:02.266556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:53:02.271824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:02.273023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:02.274590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 609/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Tue Aug 30 00:53:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:02.279082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:02.325130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:02.377451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f201b83-3cd0-4c59-8a21-5ca64cd3f68c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:02.411699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:02.417854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:02.417854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:02.417854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:02.417854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] RBAC: Adding query filter params (name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:02.417854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:02.417854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:02.423375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 627/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Tue Aug 30 00:53:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:02.429987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:02.517368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:02.533267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787f776-48b3-4db4-91f5-c5008624c6c1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:02.569579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:02.570241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:02.570530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:02.570808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:02.580818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:02.582558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:02.597493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:02.598730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:02.599812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:02.599812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:02.599812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:02.599812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 610/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:02.604643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:02.714481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5317226-e872-4971-b543-f6b5c3946b0c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:02.724275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:02.750020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:02.750639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:02.750925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:02.751203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:02.751844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:02.752372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:02.753583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:02.765472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 628/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:02.771982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:02.872894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83a744-e8b4-4d9a-9864-7612714d737f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:02.906960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:02.907572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:02.907849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:02.908122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:02.911364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:02.917092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:02.918660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:53:02.929772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:02.931506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:02.933418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 611/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:02.936906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:03.040777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5098fe-99c5-40d3-855d-17e4424706ce None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:03.076189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:03.076797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:03.077086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:03.077358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:03.078109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:03.078415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:03.079837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:03.098337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 629/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 30 00:53:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:03.104039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:03.212130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1b6e8-5a53-4e2c-91c4-e012f19c363c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:03.246237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:03.246847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:03.247128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:03.247402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:03.309364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:03.315645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:03.317369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:03.418660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 612/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:53:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:53:03.424483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:03.453007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:03.527769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb93f730-ba2c-461a-9b82-e7d773ee7c6e None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:03.564538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:03.565100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:03.565383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:03.565657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:03.575943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:03.577406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:03.592716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:03.593817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:03.594833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:03.594833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:03.594833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:03.594833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 630/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:03.600421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:03.691830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:03.697381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279de81-2fc1-406a-9cee-d9e83d6e31ec None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:03.732648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:03.733262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:03.733548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:03.733858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:03.734500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:03.734827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:03.736174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:03.748005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 613/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:03.756197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:03.834224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:03.855099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10537657-2de9-4c80-9277-53337f879ec3 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:03.888590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:03.889220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:03.889501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:03.890083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:03.897489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:03.898984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:53:03.910478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:03.917702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:03.919930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 631/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:03.921051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:04.023263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e5bf8-b514-41d3-8a79-041ccf6e47c4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:04.056720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:04.057327 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:04.057649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:04.057953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:04.058638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:04.058946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:04.060376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe160d6-e9eb-422d-a283-44efeaf11b8b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:04.073084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 614/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:53:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:04.079482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:04.183465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d928855-9a85-49e1-a41b-fb0be42100c9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:04.218093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:04.218691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:04.219811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:04.219811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:04.219987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] RBAC: Adding query filter params (role.id=89e918e9fe3a440498130d9fe2d5a464, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=38d22a2e2f374aa6bba35838ebde3b4c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:04.220233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:04.221662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf8233a-544f-4f0b-a60e-ccfff581c0b7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:04.227747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 632/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:53:04 2022] GET /identity/v3/role_assignments?user.id=38d22a2e2f374aa6bba35838ebde3b4c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=89e918e9fe3a440498130d9fe2d5a464 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:06.210217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cdd0-1b70-4786-b95d-09b5a368083e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:06.210778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cdd0-1b70-4786-b95d-09b5a368083e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:06.210778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0cdd0-1b70-4786-b95d-09b5a368083e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:06.212097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 615/1247] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:06.217226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:06.217614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:06.217816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:06.224948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:06.236829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:06.238841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:06.304237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 633/1248] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:06.567634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cdd0-1b70-4786-b95d-09b5a368083e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:06.587319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:06.671659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0cdd0-1b70-4786-b95d-09b5a368083e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:06.706950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:06.707856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:06.708133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:06.708402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:06.718362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:53:06.723610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:06.724772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:06.725650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 616/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:53:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:06.730710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:06.839040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6abeff7-86a2-46a2-9deb-bec1dc33b805 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:06.868477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:06.873363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:06.874009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:06.874286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:06.874561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:06.875218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:06.875521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:06.876890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:06.885544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 634/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:06.891553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:06.970203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:06.992881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55eee87-a793-4c5a-b7e1-819c51aa1097 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:07.026898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d873744-207b-4274-9364-94f6750a62db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:07.027534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d873744-207b-4274-9364-94f6750a62db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:07.027819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d873744-207b-4274-9364-94f6750a62db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:07.028096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d873744-207b-4274-9364-94f6750a62db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:07.039149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d873744-207b-4274-9364-94f6750a62db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:07.040604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d873744-207b-4274-9364-94f6750a62db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1d873744-207b-4274-9364-94f6750a62db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:07.054768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:07.056006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:07.057166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:07.057166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:07.057166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:07.057166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 617/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:07.061869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:07.166383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae21ea-1b99-47c2-96fc-a2c4df87cf4c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:07.199595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:07.200199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:07.200483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:07.200759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:07.201391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:07.201695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:07.203096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:07.215949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 635/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:07.221622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d873744-207b-4274-9364-94f6750a62db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:07.325535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d873744-207b-4274-9364-94f6750a62db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:07.352828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:07.360571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:07.361203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:07.361487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:07.361790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:07.369861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:07.371292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:53:07.382320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:07.383396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:07.384774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 618/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:07.389343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:07.445717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:07.490806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f548ad7d-d622-4914-9c0c-49f3c8efef30 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:07.525003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:07.525600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:07.525912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:07.526184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:07.526894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:07.527190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:07.528622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:07.545236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 636/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 30 00:53:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:07.551753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:07.654655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74349f67-b8e1-48fb-a953-2cd6278ac8a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:07.689645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:07.690284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:07.691064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:07.691402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:07.750186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:07.762828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:07.764281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:07.864433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 619/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:53:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:53:07.870461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:07.974142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0e8a4-3925-49d5-8b93-5fb4b11cd7b4 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:07.988518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:08.010112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:08.010678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:08.010962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:08.011239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:08.021671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:08.023181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:08.038043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:08.043997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:08.045197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:08.045197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:08.045197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:08.045197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 637/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:08.045503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:08.134424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:08.146756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c66080-5a5a-449a-9122-781d98a6f8de None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:08.181390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:08.182035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:08.182315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:08.182593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:08.183236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:08.183542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:08.184920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:08.196135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 620/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:08.201716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:08.303742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f445e-1a25-403b-b4e6-2d5b0dd992a9 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:08.337864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:08.338489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:08.338771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:08.339047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:08.345814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:08.347776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:53:08.359388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:08.360433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:08.361493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 638/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:53:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:08.367160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:08.471532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101cd0c-521e-4891-b663-fe8ffa33025f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:08.507791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:08.508869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:08.508869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:08.509056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:08.509710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:08.510050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:08.511486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e099a2a-7459-4f4d-bf10-bca9a4343839 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:08.523942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 621/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:53:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:08.530102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:08.556418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:08.631778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad95de-6919-4ce3-9a5a-7acbace2c832 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:08.667092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:08.667703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:08.667984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:08.668256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:08.668972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=38d22a2e2f374aa6bba35838ebde3b4c) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:08.669373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:08.670830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1a2087-0f33-43b0-914d-ff71c9560be7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:08.677205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 639/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:53:08 2022] GET /identity/v3/role_assignments?user.id=38d22a2e2f374aa6bba35838ebde3b4c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=9d7628878120443f8b9950c60d7a8671 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:10.580453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d5e64-2d83-49e3-b152-1561c24afb3f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:10.580972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d5e64-2d83-49e3-b152-1561c24afb3f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:10.581106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d5e64-2d83-49e3-b152-1561c24afb3f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:10.582656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 622/1261] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:10.587827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:10.588261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:10.588390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:10.607068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:10.608256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:10.676566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 640/1262] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:10.993124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d5e64-2d83-49e3-b152-1561c24afb3f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:11.004449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:11.094884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d5e64-2d83-49e3-b152-1561c24afb3f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:11.130098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:11.130732 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:11.131015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:11.131290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:11.141782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:53:11.147098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:11.148291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:53:11.149202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:11.153274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 623/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:11.154585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:11.178450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:11.269359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa2d3f-0cbd-4f84-ba21-030c2b1aa84c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:11.303293 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:11.303895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:11.304174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:11.304446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:11.305095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:11.305396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:11.306802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:11.314931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 641/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:53:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:11.321465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:11.390021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:11.424272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e6e175-bced-4880-b4db-caf479e4f0f2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:11.459183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:11.460569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:11.460569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:11.460569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:11.470462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:11.471974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:11.487411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:11.505516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:11.506556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:11.506556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:11.506556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:11.506556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 624/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:11.506832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:11.574873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:11.593770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a62804-9197-4d4f-b632-4d2dc143f566 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:11.626351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:11.626951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:11.627227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:11.627498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:11.628663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:11.628962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:11.630345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:11.641619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 642/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:11.647745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:11.751867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1b99ee-70ea-467b-b6e6-f956ccacaf02 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:11.775188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:11.793225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:11.793906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:11.794330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:11.794476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:11.805489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:11.807417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:53:11.820270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:11.826867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:11.827880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 625/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:11.828540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:11.941648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4959264-529c-4e34-8f1b-59f86faef98d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:11.991804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:11.992412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:11.992688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:11.992959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:11.993799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:11.994363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:11.996956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:12.011926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:12.022475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 643/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 30 00:53:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:12.034930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:12.141721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a38f27c-2c91-44a5-bec0-a295028695b4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:12.179067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:12.180333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.180333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.180333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.192504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:12.193993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:12.211539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:12.212911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:12.213890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:12.213890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:12.213890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:12.213890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 626/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:12.218757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:12.347653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2716d1-1a3c-4e9d-b6e3-6a89f097ebbb None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:12.389435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:12.390083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.390416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.390768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.391435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:12.391865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:12.393232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:12.410390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 644/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:12.416489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:12.456036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:12.535438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b23a0d-910a-4e1f-8cb4-b1fe5e0e8e66 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:12.590591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:12.591251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.591539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.591818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.599559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:12.601019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:53:12.616798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:12.617987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:12.619094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:12.619531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 627/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:53:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 00:53:12.624874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:12.693151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:12.740156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c17374-2a5a-49e6-bf41-3ccbece01822 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:12.787829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:12.788435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.788721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.789042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.789761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:12.790072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:12.791515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9afee0-0b39-4580-b5e9-cf5f1d8a8f83 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:12.805581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 645/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:53:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:12.812655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:12.908627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:12.918317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d477cfe-4f55-4938-a296-22f7d8475584 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:12.954563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ecad98-4b21-43ef-bae6-1948e0ff2bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:12.954563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecad98-4b21-43ef-bae6-1948e0ff2bfc None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:12.954563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecad98-4b21-43ef-bae6-1948e0ff2bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:12.954563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ecad98-4b21-43ef-bae6-1948e0ff2bfc None admin] PATH_INFO: `/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/38d22a2e2f374aa6bba35838ebde3b4c/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:12.956266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecad98-4b21-43ef-bae6-1948e0ff2bfc None admin] RBAC: Authorizing `identity:create_grant(project_id=8641bb2fa3db4ff79d2bb5fc288a6c81, user_id=38d22a2e2f374aa6bba35838ebde3b4c, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:12.958038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ecad98-4b21-43ef-bae6-1948e0ff2bfc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:13.177522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 628/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Tue Aug 30 00:53:12 2022] PUT /identity/v3/projects/8641bb2fa3db4ff79d2bb5fc288a6c81/users/38d22a2e2f374aa6bba35838ebde3b4c/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 00:53:15.082989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9ec4-8451-4373-ae93-4cc9bef44083 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.083544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9ec4-8451-4373-ae93-4cc9bef44083 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.083544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d9ec4-8451-4373-ae93-4cc9bef44083 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.084865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 646/1274] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:15.089903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.090333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.090473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.109624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:15.110854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:15.181052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 629/1275] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:15.455300 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d9ec4-8451-4373-ae93-4cc9bef44083 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:15.559525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d9ec4-8451-4373-ae93-4cc9bef44083 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:15.576840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:15.594314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:15.594950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.595236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.595514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.606189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 00:53:15.610951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:15.612030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:15.613721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 647/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Tue Aug 30 00:53:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:15.618353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:15.702539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:15.721274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01399d-542c-423a-8dc6-ffdbd29ae320 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:15.763726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:15.764345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.764633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.764905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.765580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:15.765921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:15.767438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:15.776626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 630/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Tue Aug 30 00:53:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:15.782677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:15.884028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5615e405-a096-480f-a0bd-271c87df9e9c None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:15.917302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:15.917956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:15.918236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:15.918506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:15.928648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:15.930169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:15.946530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:15.947802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:15.948812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:15.948812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:15.948812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:15.948812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 648/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:15.955511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:16.064984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3076053-33cb-4763-a6c7-311dbd813711 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:16.099419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:16.100010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:16.100294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:16.100568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:16.101202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:16.101506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:16.102908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:16.115509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 631/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:16.121166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:16.152127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:16.221111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94c2d8-f050-46d4-9651-9162c6dfa2fe None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:16.255316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:16.255973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:16.256250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:16.256522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:16.265413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:16.266913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 00:53:16.280211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:16.281920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:16.282993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 649/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:16.288725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:16.292921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:16.395390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51783dfd-9f7f-4dd3-a764-4765f130eb3a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:16.435723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:16.436337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:16.436621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:16.436898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:16.437622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:16.437962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:16.439396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:16.457209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 632/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Tue Aug 30 00:53:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 00:53:16.463167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:16.555220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:16.564843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682c33d-fe78-4956-b197-e9986c8dec7d None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:16.598485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:16.599092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:16.599377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:16.599654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:16.674668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:16.676759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:16.767700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:16.777996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 650/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Tue Aug 30 00:53:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 00:53:16.784608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:16.853810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:16.893266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96399bc-dd41-41b3-95c4-8ff622226bb4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:16.928406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:16.928965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:16.929249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:16.929527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:16.939397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:16.940917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 00:53:16.961313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:16.968230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 00:53:16.969324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 00:53:16.969324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 00:53:16.969324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:16.969324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 633/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 00:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:16.969620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:17.080661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a24fe6-fe4c-43ee-b907-69a822786b95 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:17.114967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:17.115590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:17.115866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:17.116137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:17.116763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:17.117062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:17.118447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:17.129553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 651/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Tue Aug 30 00:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:17.135403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:17.242634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99b8d-e714-4e7f-b424-b223f6458275 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:17.257141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:17.277669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:17.278323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:17.278602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:17.278874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:17.286042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:17.287462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 00:53:17.298412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:17.300578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 30 00:53:17.301670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 634/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Tue Aug 30 00:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:17.308009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:17.360374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:17.414872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02535ac-6abb-452f-8771-c2fb8c3386a7 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:17.449146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:17.449809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:17.450094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:17.450372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:17.451056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:17.451366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:17.452824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772aacb8-faad-4356-841f-76421cf6c257 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:17.465029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 652/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Tue Aug 30 00:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:17.474780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:17.573267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2c1f2-1a4d-40b3-86e4-c23534fd9ea2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:17.611521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:17.612170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:17.612953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:17.613237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:17.613985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] RBAC: Adding query filter params (role.id=9d7628878120443f8b9950c60d7a8671, scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81, user.id=38d22a2e2f374aa6bba35838ebde3b4c) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:17.614294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:17.615758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314cd080-c119-46d9-948e-58f84f7fd73c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:17.621709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 635/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Tue Aug 30 00:53:17 2022] GET /identity/v3/role_assignments?user.id=38d22a2e2f374aa6bba35838ebde3b4c&scope.project.id=8641bb2fa3db4ff79d2bb5fc288a6c81&role.id=9d7628878120443f8b9950c60d7a8671 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:19.595981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63885d15-f08c-4a34-ac0f-110c80867e5b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:19.597068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63885d15-f08c-4a34-ac0f-110c80867e5b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:19.597068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63885d15-f08c-4a34-ac0f-110c80867e5b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:19.598437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 653/1288] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:19.603609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:19.604021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:19.604161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:19.623097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:19.624266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:19.653190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:19.690295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 636/1289] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:19.960717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63885d15-f08c-4a34-ac0f-110c80867e5b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:20.063485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63885d15-f08c-4a34-ac0f-110c80867e5b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:20.075660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:20.100207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:20.100847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:20.101132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:20.101403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:20.102011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:20.103456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:20.113185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:20.114452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:20.115521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 654/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:53:19 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:20.121182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:20.221590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d7105-7fda-4af5-bd02-036c61466df8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:20.257319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:20.257961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:20.258245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:20.258522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:20.259152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:20.259462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:20.260834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c1a4d7-1302-46e2-a433-c4e769e09be8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:20.265947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 637/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 30 00:53:20 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:20.272106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:20.375511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97315bad-e679-43ad-a9fc-5174f98a4ad0 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:20.412673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c084f-c934-4b82-8163-81706466722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:20.413289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c084f-c934-4b82-8163-81706466722d None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:20.413572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c084f-c934-4b82-8163-81706466722d None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:20.413883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c084f-c934-4b82-8163-81706466722d None admin] PATH_INFO: `/v3/services` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:20.414519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c084f-c934-4b82-8163-81706466722d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:20.425984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c084f-c934-4b82-8163-81706466722d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:20.425984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c084f-c934-4b82-8163-81706466722d None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:20.425984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 655/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 30 00:53:20 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:22.350319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3027-07ce-4f9b-bbf9-cecf27a75184 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:22.350853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3027-07ce-4f9b-bbf9-cecf27a75184 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:22.350853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d3027-07ce-4f9b-bbf9-cecf27a75184 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:22.352206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 638/1293] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:22.357325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a31c6-0940-42dc-8d57-4fea73783cc2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:22.357762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a31c6-0940-42dc-8d57-4fea73783cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:22.357926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a31c6-0940-42dc-8d57-4fea73783cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:22.376581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3f1a31c6-0940-42dc-8d57-4fea73783cc2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:22.377876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3f1a31c6-0940-42dc-8d57-4fea73783cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:22.407490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:22.444849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 656/1294] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:22.710743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3027-07ce-4f9b-bbf9-cecf27a75184 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:22.743564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:22.815252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d3027-07ce-4f9b-bbf9-cecf27a75184 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:22.849711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2998131c-2c03-409d-9d73-a8490e835b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:22.850353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998131c-2c03-409d-9d73-a8490e835b7c None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:22.850639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998131c-2c03-409d-9d73-a8490e835b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:22.850919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2998131c-2c03-409d-9d73-a8490e835b7c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:22.851609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998131c-2c03-409d-9d73-a8490e835b7c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:22.852970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2998131c-2c03-409d-9d73-a8490e835b7c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:22.974884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 639/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Tue Aug 30 00:53:22 2022] POST /identity/v3/services => generated 246 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:53:24.905009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c69-4295-4ea9-b556-739a1874ae02 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:24.905588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c69-4295-4ea9-b556-739a1874ae02 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:24.905588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb97c69-4295-4ea9-b556-739a1874ae02 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:24.906990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 657/1296] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:24.913113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:24.913563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:24.913671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:24.932852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:24.934074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:25.018218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 640/1297] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:24 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:25.293924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c69-4295-4ea9-b556-739a1874ae02 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:25.399907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb97c69-4295-4ea9-b556-739a1874ae02 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:25.434370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:25.434995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.435274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.435548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.436117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:25.437518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:25.448303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:25.449440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:25.450351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 658/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:53:25 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:25.455604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:25.476438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:25.555484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c75bc1-1749-4307-a19f-1c9e819b96e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:25.587721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:25.588302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.588583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.588854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.589466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:25.589793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:25.591137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:25.596674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 641/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 30 00:53:25 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:25.603062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:25.635327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:25.702588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab18400-20ab-462e-8f16-870d821ab479 None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:25.737658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:25.738282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.738562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.738848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.739529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] RBAC: Adding query filter params (interface=public, service_id=ecb3b50b8cc04ddeb208d3de25f0c4c4, region_id=RegionOne) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:25.739839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:25.741205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14490d6-0990-447d-b157-5e364a171236 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:25.746228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 659/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Tue Aug 30 00:53:25 2022] GET /identity/v3/endpoints?service_id=ecb3b50b8cc04ddeb208d3de25f0c4c4&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:25.752089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:25.854216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a754a-611d-4ceb-84a0-c899332726cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:25.864406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:25.892691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beaedf6-24d5-4302-8b1c-ca8ac3948076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:25.893865 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beaedf6-24d5-4302-8b1c-ca8ac3948076 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:25.894147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beaedf6-24d5-4302-8b1c-ca8ac3948076 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:25.894420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beaedf6-24d5-4302-8b1c-ca8ac3948076 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:25.895020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beaedf6-24d5-4302-8b1c-ca8ac3948076 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:25.896390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beaedf6-24d5-4302-8b1c-ca8ac3948076 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:25.903160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 642/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Tue Aug 30 00:53:25 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 00:53:27.812762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f541f3f-9250-4f8e-b10a-025cfbf85b46 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:27.813336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f541f3f-9250-4f8e-b10a-025cfbf85b46 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:27.813336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f541f3f-9250-4f8e-b10a-025cfbf85b46 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:27.815268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 660/1302] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 00:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 00:53:27.822548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:27.822548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:27.822548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:27.840354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 00:53:27.841557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 00:53:27.912229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 643/1303] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 00:53:27 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 00:53:28.195324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f541f3f-9250-4f8e-b10a-025cfbf85b46 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:28.292544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:28.299806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f541f3f-9250-4f8e-b10a-025cfbf85b46 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:28.334275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:28.334907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:28.335186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:28.335455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:28.336022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:28.337380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 00:53:28.346048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 00:53:28.347888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application with Lock( Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self._enter() Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application return self.creator() Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application created_value = creator( Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: ERROR keystone.server.flask.application Aug 30 00:53:28.348850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 661/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Tue Aug 30 00:53:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 00:53:28.353802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:28.433923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:28.457356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689a8b9-5356-4c92-a179-fff1f3474781 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:28.492837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:28.493480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:28.493796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:28.494079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:28.494711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 00:53:28.495021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:28.496400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de449498-b303-4dff-b7d7-b42bd78d26c8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:28.502090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 644/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Tue Aug 30 00:53:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 00:53:28.508624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 00:53:28.610718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf687bc-3977-4438-a303-62e4d042720f None admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 00:53:28.646276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d227d325-ccab-4d26-9a7c-c0722b418251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 00:53:28.646883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d325-ccab-4d26-9a7c-c0722b418251 None admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 00:53:28.647164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d325-ccab-4d26-9a7c-c0722b418251 None admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 00:53:28.647440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d227d325-ccab-4d26-9a7c-c0722b418251 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 00:53:28.648098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227d325-ccab-4d26-9a7c-c0722b418251 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 00:53:28.649463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d227d325-ccab-4d26-9a7c-c0722b418251 None admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 00:53:28.750507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 662/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 00:53:28 2022] POST /identity/v3/endpoints => generated 331 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 00:53:38.618295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=30 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 00:53:38.745511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:23.746203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb09b00-3b97-43eb-bc2c-c38e49f4eec6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.746687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb09b00-3b97-43eb-bc2c-c38e49f4eec6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.746776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb09b00-3b97-43eb-bc2c-c38e49f4eec6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.754238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94f528-067a-4239-9e20-7dc9c0a675cb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:23.754664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94f528-067a-4239-9e20-7dc9c0a675cb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:23.754744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94f528-067a-4239-9e20-7dc9c0a675cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:23.759672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 645/1307] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:23 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:23.832448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4d94f528-067a-4239-9e20-7dc9c0a675cb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:23.833682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4d94f528-067a-4239-9e20-7dc9c0a675cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:23.966323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 663/1308] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:24.234731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554bd96-a817-4f88-98ca-3b8154c96c88 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.235021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554bd96-a817-4f88-98ca-3b8154c96c88 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.235304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554bd96-a817-4f88-98ca-3b8154c96c88 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.236965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 646/1309] 10.222.0.52 () {54 vars in 965 bytes} [Tue Aug 30 01:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:24.243114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bec07d-7b77-4094-8142-fc335ba32d47 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.243255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bec07d-7b77-4094-8142-fc335ba32d47 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.243589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bec07d-7b77-4094-8142-fc335ba32d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.327084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f1bec07d-7b77-4094-8142-fc335ba32d47 None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:24.328424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f1bec07d-7b77-4094-8142-fc335ba32d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:24.351181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:24.412276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 664/1310] 10.222.0.52 () {58 vars in 1062 bytes} [Tue Aug 30 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:24.421594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554bd96-a817-4f88-98ca-3b8154c96c88 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:24.423391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0554bd96-a817-4f88-98ca-3b8154c96c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:24.756014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554bd96-a817-4f88-98ca-3b8154c96c88 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:24.756014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:24.756014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:24.756014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:24.756743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:24.756743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:24.756743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:24.797826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:24.799397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2d2095-1c0a-4d43-b0f0-fbe704ea99c6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:24.820033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 647/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:27.065223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f70f-eb0e-4a45-abb1-f477288aa13f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.065825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f70f-eb0e-4a45-abb1-f477288aa13f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.065825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa19f70f-eb0e-4a45-abb1-f477288aa13f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.067562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 665/1312] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:27.072954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eec9c-fd86-421d-8ae4-e22831da1e70 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.073389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eec9c-fd86-421d-8ae4-e22831da1e70 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.073487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06eec9c-fd86-421d-8ae4-e22831da1e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.165872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c06eec9c-fd86-421d-8ae4-e22831da1e70 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:27.166995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c06eec9c-fd86-421d-8ae4-e22831da1e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:27.294420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 648/1313] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:27.602045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106c2ae-763b-47ef-84ff-13984272f47e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.602343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106c2ae-763b-47ef-84ff-13984272f47e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.602663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106c2ae-763b-47ef-84ff-13984272f47e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.604180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 666/1314] 10.222.0.52 () {54 vars in 965 bytes} [Tue Aug 30 01:02:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:27.610481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aacfa-c7b3-4312-a25e-79aa5d1d0c8f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.610840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aacfa-c7b3-4312-a25e-79aa5d1d0c8f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.611181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7aacfa-c7b3-4312-a25e-79aa5d1d0c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.623375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.699119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6d7aacfa-c7b3-4312-a25e-79aa5d1d0c8f None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:27.701222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d7aacfa-c7b3-4312-a25e-79aa5d1d0c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:27.788543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 649/1315] 10.222.0.52 () {58 vars in 1062 bytes} [Tue Aug 30 01:02:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:27.797955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106c2ae-763b-47ef-84ff-13984272f47e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:27.799629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2106c2ae-763b-47ef-84ff-13984272f47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:27.908111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106c2ae-763b-47ef-84ff-13984272f47e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:27.980871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:27.983976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:27.984487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:27.984771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:27.985046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:27.986698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:28.128805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:28.130365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66015c5b-b372-4778-8a3b-13c43304aa97 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:28.151547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 667/1316] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:30.861275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80854-ea73-439c-acd3-b79f63aba7e9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.861855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80854-ea73-439c-acd3-b79f63aba7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.861979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80854-ea73-439c-acd3-b79f63aba7e9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.863310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 650/1317] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:30.868402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cc2f6-0019-44ad-afa3-561e195a4709 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:30.868855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cc2f6-0019-44ad-afa3-561e195a4709 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:30.868948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667cc2f6-0019-44ad-afa3-561e195a4709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:30.935286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-667cc2f6-0019-44ad-afa3-561e195a4709 None None] MFA Rules not processed for user `ffd05526cecb4b7fb2137f53df2eeae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:30.936495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-667cc2f6-0019-44ad-afa3-561e195a4709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:30.988007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.032761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 668/1318] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:31.318944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b80854-ea73-439c-acd3-b79f63aba7e9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:31.320544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-15b80854-ea73-439c-acd3-b79f63aba7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:31.431549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b80854-ea73-439c-acd3-b79f63aba7e9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:31.435247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:31.510182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:31.510796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:31.511086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:31.511386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:31.514347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:31.631278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:31.632631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ba69e3-f8eb-487d-b570-511d9e9ccd2b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:31.651240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 651/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:31 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:34.594132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971018f6-c130-49cc-8ecb-bf97cc1b4b61 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.594660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971018f6-c130-49cc-8ecb-bf97cc1b4b61 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.594756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971018f6-c130-49cc-8ecb-bf97cc1b4b61 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.596191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 669/1320] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:34.601641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ab44-c638-4845-a738-ae381920b166 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:34.602089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ab44-c638-4845-a738-ae381920b166 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:34.603562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0ab44-c638-4845-a738-ae381920b166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:34.645464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:34.672372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1ae0ab44-c638-4845-a738-ae381920b166 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:34.673611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1ae0ab44-c638-4845-a738-ae381920b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:34.777316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 652/1321] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:35.068745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971018f6-c130-49cc-8ecb-bf97cc1b4b61 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:35.070320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-971018f6-c130-49cc-8ecb-bf97cc1b4b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.174904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971018f6-c130-49cc-8ecb-bf97cc1b4b61 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:35.249085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:35.249704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:35.250010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:35.250281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:35.251876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:35.260457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:35.364689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:35.366217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab62b9-a60b-4a20-98c7-0d93bd011ea9 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:35.384099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 670/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:38.662678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1e8e8-5ec3-444a-b23a-58206994989c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.663209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1e8e8-5ec3-444a-b23a-58206994989c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.663209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed1e8e8-5ec3-444a-b23a-58206994989c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.664595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 653/1323] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:38.670273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda7e8-0ad1-4e8e-996e-7e61bbaece98 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:38.670710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda7e8-0ad1-4e8e-996e-7e61bbaece98 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:38.670838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bda7e8-0ad1-4e8e-996e-7e61bbaece98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:38.742242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-73bda7e8-0ad1-4e8e-996e-7e61bbaece98 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:38.743716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-73bda7e8-0ad1-4e8e-996e-7e61bbaece98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:38.810378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:38.851931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 671/1324] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:02:39.137690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1e8e8-5ec3-444a-b23a-58206994989c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:39.139331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0ed1e8e8-5ec3-444a-b23a-58206994989c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.254649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed1e8e8-5ec3-444a-b23a-58206994989c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:39.267899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:39.333106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:39.333913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:39.334235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:39.334468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:39.336219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:39.450301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:39.451649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1519280-c4b4-4a26-86c5-b80fe90594e5 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:39.472875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 654/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:41.502626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e4319-aefc-462c-ab09-7d1db7223cef None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.503140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e4319-aefc-462c-ab09-7d1db7223cef None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.503244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03e4319-aefc-462c-ab09-7d1db7223cef None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.504661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 672/1326] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:41.509798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76254e6-4993-4150-836f-9d283e299371 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:41.510265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76254e6-4993-4150-836f-9d283e299371 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:41.510368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76254e6-4993-4150-836f-9d283e299371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:41.562877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:41.583236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d76254e6-4993-4150-836f-9d283e299371 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:41.584435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d76254e6-4993-4150-836f-9d283e299371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:41.683992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 655/1327] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:41.970930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e4319-aefc-462c-ab09-7d1db7223cef None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:41.972618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f03e4319-aefc-462c-ab09-7d1db7223cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.082745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03e4319-aefc-462c-ab09-7d1db7223cef None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:42.160727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:42.161277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:42.161547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:42.161858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:42.164671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:42.183095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:42.282099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:42.283576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbfa923-4a47-4169-b109-ddaf6219983f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:42.302154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 673/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:44.318571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb799ad-10e3-4fa3-96b7-3ff95550f89c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.319101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb799ad-10e3-4fa3-96b7-3ff95550f89c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.319222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb799ad-10e3-4fa3-96b7-3ff95550f89c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.320484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 656/1329] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:44.326368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7a05c-088b-4a7b-885a-364fffd71aff None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:44.326826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7a05c-088b-4a7b-885a-364fffd71aff None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:44.326987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe7a05c-088b-4a7b-885a-364fffd71aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:44.404405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dbe7a05c-088b-4a7b-885a-364fffd71aff None None] MFA Rules not processed for user `ffd05526cecb4b7fb2137f53df2eeae7`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:44.405610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dbe7a05c-088b-4a7b-885a-364fffd71aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.477107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:44.509839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 674/1330] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:44.795559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb799ad-10e3-4fa3-96b7-3ff95550f89c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:44.797127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dfb799ad-10e3-4fa3-96b7-3ff95550f89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:44.922504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb799ad-10e3-4fa3-96b7-3ff95550f89c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:44.945774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:45.002785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:45.003394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:45.003672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:45.003941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:45.005532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:45.116076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:45.117434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35c132c-3862-402c-822f-31f1e30b71e3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:45.135590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 657/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:47.569554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eed9f-3db2-4e64-9181-d80aa12475a3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.570133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eed9f-3db2-4e64-9181-d80aa12475a3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.570229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3eed9f-3db2-4e64-9181-d80aa12475a3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.571517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 675/1332] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:47.576934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09224f-3c5b-4c59-a62b-4e87d7e8e34d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:47.577392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09224f-3c5b-4c59-a62b-4e87d7e8e34d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:47.577499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09224f-3c5b-4c59-a62b-4e87d7e8e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:47.644369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:47.647319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5c09224f-3c5b-4c59-a62b-4e87d7e8e34d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:47.648605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5c09224f-3c5b-4c59-a62b-4e87d7e8e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:47.748024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 658/1333] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:48.031544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eed9f-3db2-4e64-9181-d80aa12475a3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:48.033093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2e3eed9f-3db2-4e64-9181-d80aa12475a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.140800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3eed9f-3db2-4e64-9181-d80aa12475a3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:48.214973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:48.215674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:48.215959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:48.216238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:48.217895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:48.244545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:48.348074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:48.349523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1bf78d-8851-4d61-8114-f163c78749fb service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:48.369308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 676/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:50.354928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b2a4d-aafc-4211-8206-9f1f019c6f9a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.356144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b2a4d-aafc-4211-8206-9f1f019c6f9a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.356144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b2a4d-aafc-4211-8206-9f1f019c6f9a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.357396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 659/1335] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:50.362585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075deb2-2825-4fa6-bd90-96ef80075465 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:50.363004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075deb2-2825-4fa6-bd90-96ef80075465 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:50.363163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075deb2-2825-4fa6-bd90-96ef80075465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:50.436557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7075deb2-2825-4fa6-bd90-96ef80075465 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:50.437821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7075deb2-2825-4fa6-bd90-96ef80075465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.516185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:50.539347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 677/1336] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:50.826709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b2a4d-aafc-4211-8206-9f1f019c6f9a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:50.828254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-767b2a4d-aafc-4211-8206-9f1f019c6f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:50.939532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b2a4d-aafc-4211-8206-9f1f019c6f9a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:50.969444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:51.016388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:51.017037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:51.017324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:51.017606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:51.019260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:51.129855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:51.131224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58b439a-603c-42ca-b5c9-390b76aab55a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:51.149156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 660/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:02:53.846709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85211a98-952c-49ed-bb22-28c2cc1c775a None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.847247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85211a98-952c-49ed-bb22-28c2cc1c775a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.847247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85211a98-952c-49ed-bb22-28c2cc1c775a None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.848650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 678/1338] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:53.853936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38852ff-3e62-48e7-917d-b7902046daba None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:53.854380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38852ff-3e62-48e7-917d-b7902046daba None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:53.854499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38852ff-3e62-48e7-917d-b7902046daba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:53.922816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b38852ff-3e62-48e7-917d-b7902046daba None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:53.924056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b38852ff-3e62-48e7-917d-b7902046daba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:53.933547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.028126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 661/1339] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:54.310918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85211a98-952c-49ed-bb22-28c2cc1c775a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:54.312482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-85211a98-952c-49ed-bb22-28c2cc1c775a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:54.419618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85211a98-952c-49ed-bb22-28c2cc1c775a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:54.495475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:54.496640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:54.496916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:54.497182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:54.498804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:54.533699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:54.608947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:54.610443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdc9d0-8397-4401-bd3f-e364234fe369 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:54.629033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 679/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:02:58.897648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6534f8-869d-454a-9701-364dbea2e44c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.898417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6534f8-869d-454a-9701-364dbea2e44c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.898417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6534f8-869d-454a-9701-364dbea2e44c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.899766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 662/1341] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:02:58.905330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909c59-f269-4dc9-8bdf-889f4e5214ba None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:58.905821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909c59-f269-4dc9-8bdf-889f4e5214ba None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:58.910047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03909c59-f269-4dc9-8bdf-889f4e5214ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:58.978652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-03909c59-f269-4dc9-8bdf-889f4e5214ba None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:02:58.979925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-03909c59-f269-4dc9-8bdf-889f4e5214ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.078397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.091198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 680/1342] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:02:59.373091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6534f8-869d-454a-9701-364dbea2e44c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:02:59.374678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4a6534f8-869d-454a-9701-364dbea2e44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.488906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6534f8-869d-454a-9701-364dbea2e44c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:02:59.529183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:02:59.566832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:02:59.567926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:02:59.568415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:02:59.568967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:02:59.571254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:02:59.687985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:02:59.689617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f910344-5499-4b1a-9672-8ff4ceb03ab8 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:02:59.709015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 663/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:02:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:02.866401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5169d-c7bc-4d32-a111-7486efe2473c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.866980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5169d-c7bc-4d32-a111-7486efe2473c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.866980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f5169d-c7bc-4d32-a111-7486efe2473c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.868288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 681/1344] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:02.874239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6359c9-f25f-4446-98a2-8d5ea5397de5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:02.874674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6359c9-f25f-4446-98a2-8d5ea5397de5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:02.874880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6359c9-f25f-4446-98a2-8d5ea5397de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:02.944018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6c6359c9-f25f-4446-98a2-8d5ea5397de5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:02.945237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6c6359c9-f25f-4446-98a2-8d5ea5397de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:02.965745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:03.051522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 664/1345] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:03.344724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5169d-c7bc-4d32-a111-7486efe2473c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:03.346294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-58f5169d-c7bc-4d32-a111-7486efe2473c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:03.461529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f5169d-c7bc-4d32-a111-7486efe2473c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:03.557825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:03.558452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:03.558824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:03.559100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:03.560885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:03.608538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:03.677929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:03.679314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39de470-2f29-4f12-81dc-812e71c73743 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:03.699859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 682/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:08.269625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c581-4ae2-4c3c-9b7a-94bae2a9768e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.270211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c581-4ae2-4c3c-9b7a-94bae2a9768e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.270211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c581-4ae2-4c3c-9b7a-94bae2a9768e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.275862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 665/1347] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:08.277668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb966ee5-47b2-4eb2-b36c-285821b7a533 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.278140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb966ee5-47b2-4eb2-b36c-285821b7a533 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.278140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb966ee5-47b2-4eb2-b36c-285821b7a533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.351142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-bb966ee5-47b2-4eb2-b36c-285821b7a533 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:08.352375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bb966ee5-47b2-4eb2-b36c-285821b7a533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:08.457962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:08.460925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 683/1348] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:08.759065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c581-4ae2-4c3c-9b7a-94bae2a9768e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:08.760724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3c93c581-4ae2-4c3c-9b7a-94bae2a9768e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:08.872543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93c581-4ae2-4c3c-9b7a-94bae2a9768e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:08.921674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:08.949547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:08.950377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:08.950718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:08.951263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:08.953247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:09.073130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:09.074758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cfe0c-4118-47e3-9039-2b45017b6d1b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:09.095599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 666/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:14.140806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66e13-2a2a-4ff7-92be-4031df088b4f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.141332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66e13-2a2a-4ff7-92be-4031df088b4f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.141442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b66e13-2a2a-4ff7-92be-4031df088b4f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.143263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 684/1350] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:03:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:14.148651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d18b9-d58c-4bcc-89ec-0ad62b652db3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:14.152868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d18b9-d58c-4bcc-89ec-0ad62b652db3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:14.152868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d18b9-d58c-4bcc-89ec-0ad62b652db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:14.255460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c54d18b9-d58c-4bcc-89ec-0ad62b652db3 None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:14.256844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c54d18b9-d58c-4bcc-89ec-0ad62b652db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:14.292583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:14.401692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 667/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:03:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:18.822203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486a76-0a98-4b1a-b7b5-b2b141a3fce1 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.822764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486a76-0a98-4b1a-b7b5-b2b141a3fce1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.822908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e486a76-0a98-4b1a-b7b5-b2b141a3fce1 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.825286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 685/1352] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:03:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:18.830162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf15b4-73c1-40ed-b9a0-1b83daefaf7b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:18.830605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf15b4-73c1-40ed-b9a0-1b83daefaf7b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:18.830704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf15b4-73c1-40ed-b9a0-1b83daefaf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:18.928987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cddf15b4-73c1-40ed-b9a0-1b83daefaf7b None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:18.930524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cddf15b4-73c1-40ed-b9a0-1b83daefaf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:19.016942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:19.054618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 668/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:03:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:20.620450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495efcac-546e-4f04-b62d-f567f12126b2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.621045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495efcac-546e-4f04-b62d-f567f12126b2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.621543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495efcac-546e-4f04-b62d-f567f12126b2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.623009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 686/1354] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:20.628448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fc623-e0d6-42cc-9c5b-25bd9e158239 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:20.628891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fc623-e0d6-42cc-9c5b-25bd9e158239 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:20.629092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400fc623-e0d6-42cc-9c5b-25bd9e158239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:20.710537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-400fc623-e0d6-42cc-9c5b-25bd9e158239 None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:20.711750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-400fc623-e0d6-42cc-9c5b-25bd9e158239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:20.824119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 669/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:03:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:24.243696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89007e1-e10e-47df-9cb1-a484d42fa3a4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.244247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89007e1-e10e-47df-9cb1-a484d42fa3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.244247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89007e1-e10e-47df-9cb1-a484d42fa3a4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.246550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 687/1356] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:24.252066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778a30c-2510-4d63-811d-e1ced7d7b0f6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.252502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778a30c-2510-4d63-811d-e1ced7d7b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.252666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778a30c-2510-4d63-811d-e1ced7d7b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.289865 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:24.326098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6778a30c-2510-4d63-811d-e1ced7d7b0f6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:24.327006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6778a30c-2510-4d63-811d-e1ced7d7b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:24.470610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 670/1357] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:24.781981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08610e83-56a5-44fe-862e-7485358e6c57 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.782609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08610e83-56a5-44fe-862e-7485358e6c57 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.783046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08610e83-56a5-44fe-862e-7485358e6c57 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.784785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 688/1358] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 30 01:03:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:24.796012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9fa73-ed10-4e00-83f3-4618b0466ffb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:24.797031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9fa73-ed10-4e00-83f3-4618b0466ffb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:24.797031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9fa73-ed10-4e00-83f3-4618b0466ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:24.889838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83b9fa73-ed10-4e00-83f3-4618b0466ffb None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:24.891066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83b9fa73-ed10-4e00-83f3-4618b0466ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:24.944595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.014886 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 671/1359] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 30 01:03:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:25.023350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08610e83-56a5-44fe-862e-7485358e6c57 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:25.025044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-08610e83-56a5-44fe-862e-7485358e6c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:25.188769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08610e83-56a5-44fe-862e-7485358e6c57 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:25.269388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:25.313481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e7f554-5752-4359-b767-331be298c677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:25.314149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7f554-5752-4359-b767-331be298c677 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:25.314427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7f554-5752-4359-b767-331be298c677 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:25.314696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e7f554-5752-4359-b767-331be298c677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:25.316413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-92e7f554-5752-4359-b767-331be298c677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:25.423938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7f554-5752-4359-b767-331be298c677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:25.425721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e7f554-5752-4359-b767-331be298c677 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:25.445413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 689/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:27.612716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd04c4-684c-4336-bce8-384d46bcb828 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.613222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd04c4-684c-4336-bce8-384d46bcb828 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.613222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd04c4-684c-4336-bce8-384d46bcb828 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.614741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 672/1361] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:27.619860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64894478-4e4f-49c9-80e5-1eb136c49ef9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:27.620461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64894478-4e4f-49c9-80e5-1eb136c49ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:27.620461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64894478-4e4f-49c9-80e5-1eb136c49ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:27.701895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-64894478-4e4f-49c9-80e5-1eb136c49ef9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:27.703499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-64894478-4e4f-49c9-80e5-1eb136c49ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:27.746352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:27.810145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 690/1362] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:28.102979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd04c4-684c-4336-bce8-384d46bcb828 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:28.104658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0ddd04c4-684c-4336-bce8-384d46bcb828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:28.301948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd04c4-684c-4336-bce8-384d46bcb828 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:28.314776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:28.434440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:28.434970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:28.435249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:28.435524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:28.437132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:28.548200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:28.550071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe576b1-f81d-41a1-bd30-04970162968d service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:28.573410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 673/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:30.992312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f7dae-2a37-4652-b59b-30ac96b37241 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:30.993298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f7dae-2a37-4652-b59b-30ac96b37241 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:30.993862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f7dae-2a37-4652-b59b-30ac96b37241 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:30.995481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 691/1364] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:31.001103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2e2b6-7408-490f-ade1-a90da2b965d7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.001920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2e2b6-7408-490f-ade1-a90da2b965d7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.002364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba2e2b6-7408-490f-ade1-a90da2b965d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.018621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.086874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fba2e2b6-7408-490f-ade1-a90da2b965d7 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:31.088978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fba2e2b6-7408-490f-ade1-a90da2b965d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:31.203776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 674/1365] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:31.506932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace5c0-3d80-440a-80bb-e9db5b00d222 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.507219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace5c0-3d80-440a-80bb-e9db5b00d222 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.507739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ace5c0-3d80-440a-80bb-e9db5b00d222 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.509281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 692/1366] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 30 01:03:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:31.515733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0314-c78e-49b2-a321-08b0fcf38ae1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:31.516018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0314-c78e-49b2-a321-08b0fcf38ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:31.516296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4f0314-c78e-49b2-a321-08b0fcf38ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:31.605438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ef4f0314-c78e-49b2-a321-08b0fcf38ae1 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:31.606714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef4f0314-c78e-49b2-a321-08b0fcf38ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:31.646367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:31.742929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 675/1367] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 30 01:03:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:31.752375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace5c0-3d80-440a-80bb-e9db5b00d222 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:31.754414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-83ace5c0-3d80-440a-80bb-e9db5b00d222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:31.931376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ace5c0-3d80-440a-80bb-e9db5b00d222 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:31.956057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.060966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:32.061640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.062003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.062166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.064141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.167973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:32.169593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07751b94-4365-404c-94ef-e6499cac88c4 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:32.190138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 693/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:32.704236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192c257-e874-44d7-9ed8-d6266375ed37 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.704821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192c257-e874-44d7-9ed8-d6266375ed37 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.704821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192c257-e874-44d7-9ed8-d6266375ed37 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.706424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 676/1369] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:32.711752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6fb6e-85bc-4f1e-b271-4712fcf28260 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:32.712048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6fb6e-85bc-4f1e-b271-4712fcf28260 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:32.712334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e6fb6e-85bc-4f1e-b271-4712fcf28260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:32.753790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:32.793277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-39e6fb6e-85bc-4f1e-b271-4712fcf28260 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:32.795366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-39e6fb6e-85bc-4f1e-b271-4712fcf28260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:32.906008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 694/1370] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:33.503657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59765a85-f527-4887-83ab-0267dc9fff87 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.503657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59765a85-f527-4887-83ab-0267dc9fff87 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.503657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59765a85-f527-4887-83ab-0267dc9fff87 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.506861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 677/1371] 10.222.0.52 () {54 vars in 964 bytes} [Tue Aug 30 01:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:33.511386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fcea1-32d9-4264-a274-2d71a3fe6632 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.511386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fcea1-32d9-4264-a274-2d71a3fe6632 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.511386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fcea1-32d9-4264-a274-2d71a3fe6632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.596143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b48fcea1-32d9-4264-a274-2d71a3fe6632 None None] MFA Rules not processed for user `fb22d91ffd224b6aa9093a440d5e4925`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:33.597389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b48fcea1-32d9-4264-a274-2d71a3fe6632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.648983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.682364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 695/1372] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Aug 30 01:03:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:33.688722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59765a85-f527-4887-83ab-0267dc9fff87 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:33.690298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-59765a85-f527-4887-83ab-0267dc9fff87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:33.805972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59765a85-f527-4887-83ab-0267dc9fff87 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:33.852397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:33.888768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:33.889568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:33.889934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:33.890217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:33.891957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:34.030224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:34.031874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a583ea-aa97-4030-a084-bca59201b16a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:34.055836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 678/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:34.445525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631ee17-3055-4d6b-817a-6f5953916e9f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.445755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631ee17-3055-4d6b-817a-6f5953916e9f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.446208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631ee17-3055-4d6b-817a-6f5953916e9f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.448125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 696/1374] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:34.453401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f661766-777a-4722-bc48-d68c6c963d82 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:34.453694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f661766-777a-4722-bc48-d68c6c963d82 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:34.454009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f661766-777a-4722-bc48-d68c6c963d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:34.534703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1f661766-777a-4722-bc48-d68c6c963d82 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:34.535919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f661766-777a-4722-bc48-d68c6c963d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:34.557133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:34.638266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 679/1375] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:34.931188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631ee17-3055-4d6b-817a-6f5953916e9f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:34.933320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1631ee17-3055-4d6b-817a-6f5953916e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:35.085496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631ee17-3055-4d6b-817a-6f5953916e9f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:35.139083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:35.207073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:35.207706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:35.207987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:35.208261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:35.209974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:35.308991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:35.310635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8a6d9-4daf-4d1c-ba0a-dc24137090e4 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:35.330559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 697/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:37.474637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd97ee8-f3b1-4b7b-bee4-42aeccc388d2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.475173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd97ee8-f3b1-4b7b-bee4-42aeccc388d2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.475173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd97ee8-f3b1-4b7b-bee4-42aeccc388d2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.476491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 680/1377] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:37.482602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6760000-ef22-42d4-bce6-ae8c5319b311 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:37.483048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6760000-ef22-42d4-bce6-ae8c5319b311 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:37.483260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6760000-ef22-42d4-bce6-ae8c5319b311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:37.558937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d6760000-ef22-42d4-bce6-ae8c5319b311 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:37.560127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d6760000-ef22-42d4-bce6-ae8c5319b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:37.567370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:37.658705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 698/1378] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:37.955423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd97ee8-f3b1-4b7b-bee4-42aeccc388d2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:37.958761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9dd97ee8-f3b1-4b7b-bee4-42aeccc388d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:38.137918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd97ee8-f3b1-4b7b-bee4-42aeccc388d2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:38.141796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:38.266628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:38.267165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:38.267493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:38.267771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:38.269390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:38.376910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:38.378724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66837f19-23a2-4801-bd79-da6b3571c69a service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:38.399581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 681/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:40.558770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9881790-ac4b-4881-b1d7-a51eadb72a97 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.559305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9881790-ac4b-4881-b1d7-a51eadb72a97 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.559629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9881790-ac4b-4881-b1d7-a51eadb72a97 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.561281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 699/1380] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:03:40.567123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de0487-c50c-422d-93a2-85ef316ad18b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:40.567616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de0487-c50c-422d-93a2-85ef316ad18b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:40.567722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de0487-c50c-422d-93a2-85ef316ad18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:40.579466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:40.640432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e0de0487-c50c-422d-93a2-85ef316ad18b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:40.641711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e0de0487-c50c-422d-93a2-85ef316ad18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:40.752828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 682/1381] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:41.053420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9881790-ac4b-4881-b1d7-a51eadb72a97 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:41.055643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b9881790-ac4b-4881-b1d7-a51eadb72a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:41.218048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.229363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9881790-ac4b-4881-b1d7-a51eadb72a97 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:41.354568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:41.354568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:41.354568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:41.354568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:41.355223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:41.466918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:41.468685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2895e09e-c772-4cf2-bd2b-a25bf3d1f7cc service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:41.480993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:41.490935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 700/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:43.716790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d0db-961c-4827-9036-c334106923a0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.717334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d0db-961c-4827-9036-c334106923a0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.717334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb9d0db-961c-4827-9036-c334106923a0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.718648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 683/1383] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:43.724253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb429d7-c672-48df-89d1-418a052120b8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:43.724726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb429d7-c672-48df-89d1-418a052120b8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:43.724825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb429d7-c672-48df-89d1-418a052120b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:43.801342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cdb429d7-c672-48df-89d1-418a052120b8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:43.802604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cdb429d7-c672-48df-89d1-418a052120b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:43.904354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 701/1384] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:44.250609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d0db-961c-4827-9036-c334106923a0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:44.252189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fb9d0db-961c-4827-9036-c334106923a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:44.347630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:44.428167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb9d0db-961c-4827-9036-c334106923a0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:44.558937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:44.559707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:44.560056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:44.560411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:44.562935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:44.626640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:44.669483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:44.671143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d23c95-d722-417c-8cf9-34210e59f4c4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:44.692687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 684/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:46.677032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f370-5534-4013-9d4e-10d0a15ed4f8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.677621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f370-5534-4013-9d4e-10d0a15ed4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.677845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f370-5534-4013-9d4e-10d0a15ed4f8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.679851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 702/1386] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:46.685650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3c3ed-7928-48ca-a4bf-bdec5c35259e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:46.686145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3c3ed-7928-48ca-a4bf-bdec5c35259e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:46.686258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff3c3ed-7928-48ca-a4bf-bdec5c35259e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:46.769297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fff3c3ed-7928-48ca-a4bf-bdec5c35259e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:46.771075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fff3c3ed-7928-48ca-a4bf-bdec5c35259e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:46.898252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 685/1387] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:03:47.202928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f370-5534-4013-9d4e-10d0a15ed4f8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:47.204470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-99e6f370-5534-4013-9d4e-10d0a15ed4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:47.295221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:47.388426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f370-5534-4013-9d4e-10d0a15ed4f8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:47.512825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:47.513520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:47.513848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:47.514135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:47.515830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:47.565361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:47.627071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:47.628698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e013fc4-55b5-4c1a-86a1-3e96ab3f0ca9 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:47.653084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 703/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:49.845958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a144dd-013a-4efa-ade7-c775a047fd6e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.846557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a144dd-013a-4efa-ade7-c775a047fd6e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.846557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a144dd-013a-4efa-ade7-c775a047fd6e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.848129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 686/1389] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:49.854328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880f61e-71d3-4dc3-999e-83ce93c77265 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:49.854809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880f61e-71d3-4dc3-999e-83ce93c77265 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:49.855005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880f61e-71d3-4dc3-999e-83ce93c77265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:49.922765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1880f61e-71d3-4dc3-999e-83ce93c77265 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:49.923969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1880f61e-71d3-4dc3-999e-83ce93c77265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.023932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 704/1390] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:50.312861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a144dd-013a-4efa-ade7-c775a047fd6e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:50.314546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-90a144dd-013a-4efa-ade7-c775a047fd6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.349482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:50.487901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a144dd-013a-4efa-ade7-c775a047fd6e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:50.601188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:50.606350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:50.606967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:50.607253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:50.607542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:50.609143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:50.703240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:50.704967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4761a-8fb9-4b32-ae13-57526f558121 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:50.723760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 687/1391] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:51.412564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ec6f-bcbb-4840-86a0-e94224645ce8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.413120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ec6f-bcbb-4840-86a0-e94224645ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.413120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0ec6f-bcbb-4840-86a0-e94224645ce8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.414501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 705/1392] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:51.420439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536c18a-8942-474b-b321-f465ffbcd7b0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.420887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536c18a-8942-474b-b321-f465ffbcd7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.421036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536c18a-8942-474b-b321-f465ffbcd7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.494499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b536c18a-8942-474b-b321-f465ffbcd7b0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:51.495801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b536c18a-8942-474b-b321-f465ffbcd7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:51.557148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:51.595857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 688/1393] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:51.895250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5f6d2-a68f-495f-8cb5-f9b158f0965f None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.895537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5f6d2-a68f-495f-8cb5-f9b158f0965f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.895872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5f6d2-a68f-495f-8cb5-f9b158f0965f None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.897209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 706/1394] 10.222.0.52 () {54 vars in 967 bytes} [Tue Aug 30 01:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:51.902811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3905a0-bc05-4ee0-a3f7-b560ce2a3bc9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:51.903105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3905a0-bc05-4ee0-a3f7-b560ce2a3bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:51.903374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3905a0-bc05-4ee0-a3f7-b560ce2a3bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:51.983471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7a3905a0-bc05-4ee0-a3f7-b560ce2a3bc9 None None] MFA Rules not processed for user `d0f8fc283cce4eff8126074e2a2c39fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:51.984690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7a3905a0-bc05-4ee0-a3f7-b560ce2a3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.063589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 689/1395] 10.222.0.52 () {58 vars in 1064 bytes} [Tue Aug 30 01:03:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:52.071090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5f6d2-a68f-495f-8cb5-f9b158f0965f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:52.072790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-74b5f6d2-a68f-495f-8cb5-f9b158f0965f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.086700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.184303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5f6d2-a68f-495f-8cb5-f9b158f0965f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:52.259914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:52.260527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.260805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.261076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.262723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:52.332011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:52.373748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:52.375219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a3bf78-7e4a-43cd-8e00-cdfebf8f7934 service glance] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:52.396118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 707/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:52.989699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de8757-ea9c-47d4-b9bb-b3870e10d4da None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.990241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de8757-ea9c-47d4-b9bb-b3870e10d4da None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.990336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de8757-ea9c-47d4-b9bb-b3870e10d4da None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:52.991707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 690/1397] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:52.998086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:52.998513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:52.998871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.074081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:53.075335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.198510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 708/1398] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:53.321839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c32691-49b9-4799-8cb3-5962b538f5d3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.322298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c32691-49b9-4799-8cb3-5962b538f5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.322673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c32691-49b9-4799-8cb3-5962b538f5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.394173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-48c32691-49b9-4799-8cb3-5962b538f5d3 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:53.395404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48c32691-49b9-4799-8cb3-5962b538f5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.405805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.491674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 691/1399] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:03:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:53.495370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.496581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.526497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.535252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5abdf-1bb9-4624-a8e0-923902eb873d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.540203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5abdf-1bb9-4624-a8e0-923902eb873d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.540203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b5abdf-1bb9-4624-a8e0-923902eb873d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.540203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 692/1400] 10.222.0.52 () {54 vars in 943 bytes} [Tue Aug 30 01:03:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:53.544495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.544785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.545066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.619881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a98a3c6-4c48-4f7d-8310-5d4cdb4f6861 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.630329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] MFA Rules not processed for user `6a7f47fa1a8e4902a1a68219366274c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:53.631558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.722659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.740351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.740351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.740351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.740351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.741176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.749253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 693/1401] 10.222.0.52 () {58 vars in 1040 bytes} [Tue Aug 30 01:03:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:53.759949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:53.767914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:53.772286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a4097-5df8-4d6e-a4ab-5ca10b92fea2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:53.843718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.845326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7789c-578e-4e0d-b0a5-393af3f285e2 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.864495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 709/1402] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:03:53.885514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:53.893625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:53.893625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:53.893625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:53.893625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:53.976473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:53.978247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d9976-3d34-47fd-83e8-275c8bac3ab0 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:53.986856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 694/1403] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:03:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:03:56.068186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e504998-eb5f-4f71-bcc9-38ecfec3a537 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.070102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e504998-eb5f-4f71-bcc9-38ecfec3a537 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.070102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e504998-eb5f-4f71-bcc9-38ecfec3a537 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.082560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 710/1404] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:56.082827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c4a61-41c6-483a-a45d-48cc4e794d76 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.082827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c4a61-41c6-483a-a45d-48cc4e794d76 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.082827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7c4a61-41c6-483a-a45d-48cc4e794d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.093454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.169463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fb7c4a61-41c6-483a-a45d-48cc4e794d76 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:56.170821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fb7c4a61-41c6-483a-a45d-48cc4e794d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:56.287189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 695/1405] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:56.604489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e504998-eb5f-4f71-bcc9-38ecfec3a537 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:56.607114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2e504998-eb5f-4f71-bcc9-38ecfec3a537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:56.782186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:56.784404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e504998-eb5f-4f71-bcc9-38ecfec3a537 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:03:56.913383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:03:56.914390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:56.914755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:56.915633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:56.917499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:57.032518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:03:57.034654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d95a4-da3f-4a7f-9872-1f3e7334a5d1 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:03:57.063738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 711/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:03:59.284147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbb0b1-099a-46bf-9873-f4df6c7a411d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.284722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbb0b1-099a-46bf-9873-f4df6c7a411d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.284722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbb0b1-099a-46bf-9873-f4df6c7a411d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.286099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 696/1407] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:03:59.291744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9c1a9-2514-48d9-b261-37184f1a88bb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:03:59.292445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9c1a9-2514-48d9-b261-37184f1a88bb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:03:59.292445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb9c1a9-2514-48d9-b261-37184f1a88bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:03:59.296715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.367369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-adb9c1a9-2514-48d9-b261-37184f1a88bb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:03:59.368621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-adb9c1a9-2514-48d9-b261-37184f1a88bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:59.492561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 712/1408] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:03:59.789995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbb0b1-099a-46bf-9873-f4df6c7a411d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:03:59.791760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e6dbb0b1-099a-46bf-9873-f4df6c7a411d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:03:59.901870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:03:59.969256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbb0b1-099a-46bf-9873-f4df6c7a411d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:00.114750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:00.115389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:00.115675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:00.115959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:00.117673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:00.178969 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:00.225871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:00.228413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c01643-8225-44de-84c3-139e95a8289f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:00.249150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 697/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:03:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:09.490805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=64 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:10.292714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=23 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:17.614426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6087d9-2102-471f-a0e5-234101356c19 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.614995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6087d9-2102-471f-a0e5-234101356c19 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.614995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6087d9-2102-471f-a0e5-234101356c19 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.616334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 713/1410] 10.222.0.52 () {54 vars in 918 bytes} [Tue Aug 30 01:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:17.622493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693931-1056-431f-b735-e5e0fe6af17e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.623097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693931-1056-431f-b735-e5e0fe6af17e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.623097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693931-1056-431f-b735-e5e0fe6af17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.702350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9c693931-1056-431f-b735-e5e0fe6af17e None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:17.703487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c693931-1056-431f-b735-e5e0fe6af17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:17.815549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 698/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Aug 30 01:04:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:17.830795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51128456-4b66-4236-9db8-084088d5f7db None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.831085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51128456-4b66-4236-9db8-084088d5f7db None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.831367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51128456-4b66-4236-9db8-084088d5f7db None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.832765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 714/1412] 10.222.0.52 () {54 vars in 918 bytes} [Tue Aug 30 01:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:17.838631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b9633-c8c0-4701-89c5-4efdee454459 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:17.838924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b9633-c8c0-4701-89c5-4efdee454459 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:17.839202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b9633-c8c0-4701-89c5-4efdee454459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:17.915065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:17.917820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-832b9633-c8c0-4701-89c5-4efdee454459 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:17.919060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-832b9633-c8c0-4701-89c5-4efdee454459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:18.042159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 699/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Tue Aug 30 01:04:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:04:20.483917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679078b-89f8-49e9-823c-8db255caa403 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.484440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679078b-89f8-49e9-823c-8db255caa403 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.484440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a679078b-89f8-49e9-823c-8db255caa403 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.485717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:20.491456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1b595-1836-4483-8bbf-14cddcb734af None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:20.491950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1b595-1836-4483-8bbf-14cddcb734af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:20.492091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1b595-1836-4483-8bbf-14cddcb734af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:20.562227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bee1b595-1836-4483-8bbf-14cddcb734af None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:20.563443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bee1b595-1836-4483-8bbf-14cddcb734af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:20.619193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:20.662849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:20.957262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679078b-89f8-49e9-823c-8db255caa403 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:20.958869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a679078b-89f8-49e9-823c-8db255caa403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:21.113862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a679078b-89f8-49e9-823c-8db255caa403 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:21.198554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:21.238912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:21.239683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:21.240315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:21.240591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:21.242810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:21.335193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:21.337493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf7aaf4-f7cd-41c6-8f86-2318fe0281bc service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:21.356329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 716/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:22.252303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f7ead-6a55-45db-a5c0-e8dbca4de907 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.252849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f7ead-6a55-45db-a5c0-e8dbca4de907 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.253138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f7ead-6a55-45db-a5c0-e8dbca4de907 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.254670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:22.260011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d825-4385-4cfa-8318-baa61c32c958 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.260460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d825-4385-4cfa-8318-baa61c32c958 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.260572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731d825-4385-4cfa-8318-baa61c32c958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.336888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6731d825-4385-4cfa-8318-baa61c32c958 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:22.338155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6731d825-4385-4cfa-8318-baa61c32c958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:22.365321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:22.437130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:22.598032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f7ead-6a55-45db-a5c0-e8dbca4de907 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:22.599597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a1f7ead-6a55-45db-a5c0-e8dbca4de907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:22.754235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f7ead-6a55-45db-a5c0-e8dbca4de907 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:22.796921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:22.874917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:22.875463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:22.875746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:22.876017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:22.877661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:22.973271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:22.974925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe4ea1-f215-4dcf-aeb9-13f7391284ad service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:22.993128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 702/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:04:23.922192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4db27-d61b-4c4e-9eac-be27a1bd07ea None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.922788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4db27-d61b-4c4e-9eac-be27a1bd07ea None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.922788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4db27-d61b-4c4e-9eac-be27a1bd07ea None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:23.924090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:23.929062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f622-06ed-46fd-a0b8-a43964ebcb3f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:23.929604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f622-06ed-46fd-a0b8-a43964ebcb3f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:23.929802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7f622-06ed-46fd-a0b8-a43964ebcb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.004155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.033435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bdc7f622-06ed-46fd-a0b8-a43964ebcb3f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:24.035345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bdc7f622-06ed-46fd-a0b8-a43964ebcb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:24.163901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:24.376077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4db27-d61b-4c4e-9eac-be27a1bd07ea None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:24.377784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7fa4db27-d61b-4c4e-9eac-be27a1bd07ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:24.554037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4db27-d61b-4c4e-9eac-be27a1bd07ea None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:24.568966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:24.695918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:24.696840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:24.697125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:24.697488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:24.699316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:24.805386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:24.807043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eed704c-a555-4dfc-ba92-4e4bf236cf90 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:24.825983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 719/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:25.698875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff28e0a-2875-45c0-b014-98ffeadaf1b3 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.699425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff28e0a-2875-45c0-b014-98ffeadaf1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.699425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff28e0a-2875-45c0-b014-98ffeadaf1b3 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.700845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:25.706267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b315078-8b58-4ccf-b8f4-205007ae9487 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:25.706722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b315078-8b58-4ccf-b8f4-205007ae9487 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:25.706835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b315078-8b58-4ccf-b8f4-205007ae9487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:25.727030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:25.786138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4b315078-8b58-4ccf-b8f4-205007ae9487 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:25.787397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4b315078-8b58-4ccf-b8f4-205007ae9487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:25.885087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:04:26.040238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff28e0a-2875-45c0-b014-98ffeadaf1b3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:26.041882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aff28e0a-2875-45c0-b014-98ffeadaf1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:26.153458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:26.191969 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff28e0a-2875-45c0-b014-98ffeadaf1b3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:26.324300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:26.324842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:26.325123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:26.325395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:26.327029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:26.406289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:26.420920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:26.422577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc344591-f5ac-419f-a605-b0177179b853 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:26.441965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 705/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:27.310296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf74f6-6b1b-412c-8ecf-8d06ebac08e3 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.311108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf74f6-6b1b-412c-8ecf-8d06ebac08e3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.311108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf74f6-6b1b-412c-8ecf-8d06ebac08e3 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.312467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:27.317569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e99ec9-816c-4e60-b213-d64798b26948 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.318022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e99ec9-816c-4e60-b213-d64798b26948 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.318160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e99ec9-816c-4e60-b213-d64798b26948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.395929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-97e99ec9-816c-4e60-b213-d64798b26948 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:27.397199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-97e99ec9-816c-4e60-b213-d64798b26948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:27.495138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:27.646612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf74f6-6b1b-412c-8ecf-8d06ebac08e3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:27.648092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-85bf74f6-6b1b-412c-8ecf-8d06ebac08e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:27.748971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:27.821275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf74f6-6b1b-412c-8ecf-8d06ebac08e3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:27.952668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:27.953306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:27.953586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:27.953896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:27.955527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:28.004802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:28.048089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:28.049926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4563e5-012c-49ac-8c8c-a07371bd6d58 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:28.071043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 722/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:29.015195 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c725c9-2953-4845-af45-ddf16dd82c6a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.015872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c725c9-2953-4845-af45-ddf16dd82c6a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.015872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c725c9-2953-4845-af45-ddf16dd82c6a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.017104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:29.022061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c9b89-8c53-4843-8aa5-384643adcd01 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.022466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c9b89-8c53-4843-8aa5-384643adcd01 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.022710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c9b89-8c53-4843-8aa5-384643adcd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.100272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a29c9b89-8c53-4843-8aa5-384643adcd01 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:29.101527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a29c9b89-8c53-4843-8aa5-384643adcd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:29.199848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:29.351143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c725c9-2953-4845-af45-ddf16dd82c6a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:29.352910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a0c725c9-2953-4845-af45-ddf16dd82c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:29.404411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.504712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c725c9-2953-4845-af45-ddf16dd82c6a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:29.743148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:29.744116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:29.744615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:29.745132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:29.747420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:29.754744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:29.838181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:29.839781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610585b-0ec8-4cbe-bf86-5f6320ea6069 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:29.858262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 708/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:30.743073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ae24d-fc19-45ba-9c0d-b39ef7f2f7d4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.743605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ae24d-fc19-45ba-9c0d-b39ef7f2f7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.743605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ae24d-fc19-45ba-9c0d-b39ef7f2f7d4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.744911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:30.749927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228edc-f866-4a57-8720-268840242bbc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:30.750390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228edc-f866-4a57-8720-268840242bbc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:30.750498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab228edc-f866-4a57-8720-268840242bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:30.828819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ab228edc-f866-4a57-8720-268840242bbc None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:30.832592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ab228edc-f866-4a57-8720-268840242bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:30.899215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:30.927166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:31.078441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ae24d-fc19-45ba-9c0d-b39ef7f2f7d4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:31.079971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3c3ae24d-fc19-45ba-9c0d-b39ef7f2f7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:31.097281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:31.227112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ae24d-fc19-45ba-9c0d-b39ef7f2f7d4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:31.318679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:31.344240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:31.344778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:31.345062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:31.345341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:31.346991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:31.440657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:31.442314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e24b58-7ebd-404d-a75c-f4b162772683 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:31.460310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 725/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:32.364600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f06841-68eb-4402-a078-0f0cbd93af75 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:32.365172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f06841-68eb-4402-a078-0f0cbd93af75 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:32.365172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f06841-68eb-4402-a078-0f0cbd93af75 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:32.366519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:32.371507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd562-f5d0-47f1-9fc1-5829d3120e84 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:32.371963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd562-f5d0-47f1-9fc1-5829d3120e84 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:32.372059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bdd562-f5d0-47f1-9fc1-5829d3120e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:32.454909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-77bdd562-f5d0-47f1-9fc1-5829d3120e84 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:32.456138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-77bdd562-f5d0-47f1-9fc1-5829d3120e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:32.498893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:32.556367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:32.712994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f06841-68eb-4402-a078-0f0cbd93af75 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:32.714473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b7f06841-68eb-4402-a078-0f0cbd93af75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:32.864566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f06841-68eb-4402-a078-0f0cbd93af75 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:32.920461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:32.980589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:32.981112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:32.981390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:32.981660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:32.983283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:33.072785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:33.074399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b654dda-55c0-4d4b-9193-fc38beb23673 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:33.092171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 711/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:04:33.979852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09274b8-b96d-478b-bbdc-fe28056ac12a None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:33.980443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09274b8-b96d-478b-bbdc-fe28056ac12a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:33.980443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09274b8-b96d-478b-bbdc-fe28056ac12a None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:33.981753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:33.986793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af571f18-d50d-4584-b88b-b33af3c34208 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:33.987220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af571f18-d50d-4584-b88b-b33af3c34208 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:33.987355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af571f18-d50d-4584-b88b-b33af3c34208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:34.066254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-af571f18-d50d-4584-b88b-b33af3c34208 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:34.067479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-af571f18-d50d-4584-b88b-b33af3c34208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:34.069502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:34.166522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:34.317164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09274b8-b96d-478b-bbdc-fe28056ac12a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:34.318731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a09274b8-b96d-478b-bbdc-fe28056ac12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:34.471557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09274b8-b96d-478b-bbdc-fe28056ac12a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:34.499612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:34.588382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:34.588892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:34.589174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:34.589447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:34.591090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:34.684757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:34.686658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27bdaf-a196-4321-8dda-883e633d8d68 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:34.705433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 728/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:36.620498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cae99-4739-4594-abf1-186b66e50a4e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:36.621065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cae99-4739-4594-abf1-186b66e50a4e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:36.621065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cae99-4739-4594-abf1-186b66e50a4e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:36.622453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:36.627848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b9b53-b414-416a-9d2c-b4b0cbe0c55b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:36.628258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b9b53-b414-416a-9d2c-b4b0cbe0c55b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:36.628397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b9b53-b414-416a-9d2c-b4b0cbe0c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:36.672760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:36.699738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ec9b9b53-b414-416a-9d2c-b4b0cbe0c55b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:36.700957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ec9b9b53-b414-416a-9d2c-b4b0cbe0c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:36.801816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:37.091200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cae99-4739-4594-abf1-186b66e50a4e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:37.092767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-956cae99-4739-4594-abf1-186b66e50a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:37.198465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cae99-4739-4594-abf1-186b66e50a4e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:37.230357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:37.273164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:37.273800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:37.274094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:37.274372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:37.275984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:37.385939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:37.387296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d958329-2851-49b8-b56e-2c4da8ea4b58 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:37.406736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 714/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:39.514703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e193d9-dcf0-4f41-b32d-03f5f9c8ce4e None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:39.515233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e193d9-dcf0-4f41-b32d-03f5f9c8ce4e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:39.515353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e193d9-dcf0-4f41-b32d-03f5f9c8ce4e None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:39.516683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:39.522197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abce7c-57d4-46b5-861f-ef4c3ea6c528 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:39.522653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abce7c-57d4-46b5-861f-ef4c3ea6c528 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:39.522974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60abce7c-57d4-46b5-861f-ef4c3ea6c528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:39.590006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-60abce7c-57d4-46b5-861f-ef4c3ea6c528 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:39.591238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-60abce7c-57d4-46b5-861f-ef4c3ea6c528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:39.604604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:39.689530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:39.979424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e193d9-dcf0-4f41-b32d-03f5f9c8ce4e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:39.980993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d4e193d9-dcf0-4f41-b32d-03f5f9c8ce4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:40.085854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e193d9-dcf0-4f41-b32d-03f5f9c8ce4e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:40.098470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:40.166957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:40.167563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:40.167841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:40.168109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:40.169710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:40.283879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:40.285247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62128577-ff1e-4278-b94d-bf805721d8fa service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:40.304823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 731/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:42.644525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b5e83-bab6-4ae1-a651-275633eda660 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:42.645080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b5e83-bab6-4ae1-a651-275633eda660 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:42.645080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b5e83-bab6-4ae1-a651-275633eda660 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:42.646523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:42.652303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab4be9-4f15-4c45-a800-ea558093d586 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:42.652758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab4be9-4f15-4c45-a800-ea558093d586 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:42.652929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ab4be9-4f15-4c45-a800-ea558093d586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:42.704883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:42.723663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e1ab4be9-4f15-4c45-a800-ea558093d586 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:42.724877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e1ab4be9-4f15-4c45-a800-ea558093d586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:42.824321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:43.110989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b5e83-bab6-4ae1-a651-275633eda660 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:43.112629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ae2b5e83-bab6-4ae1-a651-275633eda660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:43.217721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b5e83-bab6-4ae1-a651-275633eda660 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:43.259702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:43.293516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48292466-4530-436e-8a60-248f9720665c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:43.294167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292466-4530-436e-8a60-248f9720665c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:43.294447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292466-4530-436e-8a60-248f9720665c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:43.294724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48292466-4530-436e-8a60-248f9720665c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:43.296329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48292466-4530-436e-8a60-248f9720665c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:43.409399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48292466-4530-436e-8a60-248f9720665c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:43.410907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48292466-4530-436e-8a60-248f9720665c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:43.430528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 717/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:45.501582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509bbe2-ef62-4364-89ba-bb11533eea34 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:45.502150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509bbe2-ef62-4364-89ba-bb11533eea34 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:45.502269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509bbe2-ef62-4364-89ba-bb11533eea34 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:45.503528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:45.508829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd470d6e-b7a1-4680-b75e-3831f3366e2c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:45.509256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd470d6e-b7a1-4680-b75e-3831f3366e2c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:45.509374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd470d6e-b7a1-4680-b75e-3831f3366e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:45.577042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bd470d6e-b7a1-4680-b75e-3831f3366e2c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:45.578300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bd470d6e-b7a1-4680-b75e-3831f3366e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:45.599117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:45.677540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:45.967752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509bbe2-ef62-4364-89ba-bb11533eea34 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:45.969283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f509bbe2-ef62-4364-89ba-bb11533eea34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:46.074007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509bbe2-ef62-4364-89ba-bb11533eea34 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:46.094232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:46.148892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:46.149506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:46.149886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:46.150169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:46.151779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:46.263094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:46.264451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff03c10b-bb6a-4737-b377-573f61d9f9b0 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:46.284207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 734/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:48.355291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc01dc-079a-458f-8ea2-29fca1290a36 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:48.355873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc01dc-079a-458f-8ea2-29fca1290a36 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:48.355873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dc01dc-079a-458f-8ea2-29fca1290a36 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:48.357183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:48.362736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5e65f-6d88-4e41-83fd-d5f7a82079a0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:48.363327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5e65f-6d88-4e41-83fd-d5f7a82079a0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:48.363327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5e65f-6d88-4e41-83fd-d5f7a82079a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:48.429071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:48.432246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9ff5e65f-6d88-4e41-83fd-d5f7a82079a0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:48.433520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9ff5e65f-6d88-4e41-83fd-d5f7a82079a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:48.539254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:48.829416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc01dc-079a-458f-8ea2-29fca1290a36 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:48.843276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87dc01dc-079a-458f-8ea2-29fca1290a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:48.940474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dc01dc-079a-458f-8ea2-29fca1290a36 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:48.990056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:49.016230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:49.016890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:49.017165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:49.017435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:49.019083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:49.130526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:49.132077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5667ab-bf17-4489-9ef2-c3de5f2daf0b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:49.150852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 720/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:51.198129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062da3-f122-4492-a8a9-a9dbd558a9f5 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.198731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062da3-f122-4492-a8a9-a9dbd558a9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.198731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55062da3-f122-4492-a8a9-a9dbd558a9f5 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.200006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:51.205302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaef48-8fe2-4617-bee0-73b5a9f89793 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.205890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaef48-8fe2-4617-bee0-73b5a9f89793 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.205890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edaef48-8fe2-4617-bee0-73b5a9f89793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.273804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7edaef48-8fe2-4617-bee0-73b5a9f89793 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:51.275026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7edaef48-8fe2-4617-bee0-73b5a9f89793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:51.303983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:51.373515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:51.659450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55062da3-f122-4492-a8a9-a9dbd558a9f5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:51.661018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-55062da3-f122-4492-a8a9-a9dbd558a9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:51.768566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55062da3-f122-4492-a8a9-a9dbd558a9f5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:51.797172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:51.843888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:51.844690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:51.845139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:51.845455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:51.847498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:51.964604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:51.966109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1470a9df-4dd1-40d7-8a72-507f4fb34b73 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:51.984773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 737/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:53.987024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3949d4-ec28-4fe5-9f8d-eaa1a31f64c2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:53.987634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3949d4-ec28-4fe5-9f8d-eaa1a31f64c2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:53.987634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3949d4-ec28-4fe5-9f8d-eaa1a31f64c2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:53.988953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:53.995102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d815e1-6755-475c-972b-fbe87145f147 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:53.995737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d815e1-6755-475c-972b-fbe87145f147 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:53.995737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d815e1-6755-475c-972b-fbe87145f147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:54.076471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c1d815e1-6755-475c-972b-fbe87145f147 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:54.077704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c1d815e1-6755-475c-972b-fbe87145f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:54.086494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:54.177751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:54.463583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3949d4-ec28-4fe5-9f8d-eaa1a31f64c2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:54.465188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d3949d4-ec28-4fe5-9f8d-eaa1a31f64c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:54.577763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3949d4-ec28-4fe5-9f8d-eaa1a31f64c2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:54.638649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:54.657864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:54.658480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:54.658764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:54.659041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:54.660701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:54.772659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:54.774048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e08767-6755-4f7f-abc8-1359af14544d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:54.793142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 723/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:56.768307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f060bd4-cfea-4015-a30a-b63bcf1eeae5 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:56.768876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f060bd4-cfea-4015-a30a-b63bcf1eeae5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:56.768876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f060bd4-cfea-4015-a30a-b63bcf1eeae5 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:56.770321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:04:56.776509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03040cbd-b943-4d12-a3ee-c5ec562143b3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:56.777045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03040cbd-b943-4d12-a3ee-c5ec562143b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:56.777045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03040cbd-b943-4d12-a3ee-c5ec562143b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:56.846701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-03040cbd-b943-4d12-a3ee-c5ec562143b3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:56.847927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-03040cbd-b943-4d12-a3ee-c5ec562143b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:56.887496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:56.951582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:57.235971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f060bd4-cfea-4015-a30a-b63bcf1eeae5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:57.237614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4f060bd4-cfea-4015-a30a-b63bcf1eeae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:57.347953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f060bd4-cfea-4015-a30a-b63bcf1eeae5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:04:57.390000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:57.430640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:04:57.431253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:57.431557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:57.431835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:57.433431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:57.549541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:04:57.551038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698e22d-ab8f-4233-8f24-29728479aefa service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:04:57.572164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 740/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:04:59.496757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935a8f-fbf2-4244-a66f-79c934e63daa None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:59.497335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935a8f-fbf2-4244-a66f-79c934e63daa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:59.497441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8935a8f-fbf2-4244-a66f-79c934e63daa None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:59.498764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:04:59.503936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e91c-970e-4878-bad8-88520086f774 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:04:59.504362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e91c-970e-4878-bad8-88520086f774 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:04:59.504502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a3e91c-970e-4878-bad8-88520086f774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:04:59.571215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-27a3e91c-970e-4878-bad8-88520086f774 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:04:59.572463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-27a3e91c-970e-4878-bad8-88520086f774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:04:59.591973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:04:59.673904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:04:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:04:59.959931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935a8f-fbf2-4244-a66f-79c934e63daa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:04:59.961522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f8935a8f-fbf2-4244-a66f-79c934e63daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:00.068461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8935a8f-fbf2-4244-a66f-79c934e63daa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:00.137388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:00.144445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:00.145296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:00.145574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:00.145884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:00.148235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:00.258709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:00.260049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77688c-d518-4f26-9113-1eb03487b7b3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:00.279056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 726/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:02.212269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e28d4-3e3a-4e62-a8c9-8b302147c4e8 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:02.212845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e28d4-3e3a-4e62-a8c9-8b302147c4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:02.212845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e28d4-3e3a-4e62-a8c9-8b302147c4e8 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:02.214195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:02.219345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee2ed8-cb04-47d4-9970-081224fec264 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:02.219761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee2ed8-cb04-47d4-9970-081224fec264 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:02.219918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ee2ed8-cb04-47d4-9970-081224fec264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:02.290465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83ee2ed8-cb04-47d4-9970-081224fec264 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:02.291688 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83ee2ed8-cb04-47d4-9970-081224fec264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:02.339879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:02.388379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:02.676351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e28d4-3e3a-4e62-a8c9-8b302147c4e8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:02.677992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f29e28d4-3e3a-4e62-a8c9-8b302147c4e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:02.789461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e28d4-3e3a-4e62-a8c9-8b302147c4e8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:02.959097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:02.985541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:02.986172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:02.986453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:02.986722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:02.988305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:03.099579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:03.101545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e267143a-9792-404d-b6a7-b542dfac93a5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:03.119267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 743/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:12.386531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=22 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:13.117014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=66 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:22.840109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb80c8-7bed-4e3d-be3b-9a2d6ab489bb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:22.840657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb80c8-7bed-4e3d-be3b-9a2d6ab489bb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:22.840657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb80c8-7bed-4e3d-be3b-9a2d6ab489bb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:22.842057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 728/1471] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:05:22.847478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b320d-928e-4652-870b-63f0fa238e80 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:22.847968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b320d-928e-4652-870b-63f0fa238e80 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:22.848087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b320d-928e-4652-870b-63f0fa238e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:22.916867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-240b320d-928e-4652-870b-63f0fa238e80 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:22.918123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-240b320d-928e-4652-870b-63f0fa238e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:23.028635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 744/1472] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:23.327180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef8be-0d99-47ba-97d7-8f42ab4a320f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:23.327473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef8be-0d99-47ba-97d7-8f42ab4a320f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:23.327752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671ef8be-0d99-47ba-97d7-8f42ab4a320f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:23.329085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 729/1473] 10.222.0.52 () {54 vars in 967 bytes} [Tue Aug 30 01:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:23.335314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807fd34-445e-4ee0-a618-0493d1430404 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:23.335605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807fd34-445e-4ee0-a618-0493d1430404 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:23.335880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e807fd34-445e-4ee0-a618-0493d1430404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:23.413766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e807fd34-445e-4ee0-a618-0493d1430404 None None] MFA Rules not processed for user `d0f8fc283cce4eff8126074e2a2c39fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:23.415611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e807fd34-445e-4ee0-a618-0493d1430404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:23.437848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:23.495691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 745/1474] 10.222.0.52 () {58 vars in 1064 bytes} [Tue Aug 30 01:05:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:23.503454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef8be-0d99-47ba-97d7-8f42ab4a320f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:23.504966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-671ef8be-0d99-47ba-97d7-8f42ab4a320f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:23.610025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671ef8be-0d99-47ba-97d7-8f42ab4a320f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:23.684422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:23.685036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:23.685322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:23.685601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:23.687262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:23.742423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:23.797883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:23.799325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f7e564-44e9-48fb-ba11-8c7327c5f5e6 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:23.818604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 730/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:25.879581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147fd2b-0749-4212-a9bb-6f2283caf4fe None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:25.880666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147fd2b-0749-4212-a9bb-6f2283caf4fe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:25.880795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147fd2b-0749-4212-a9bb-6f2283caf4fe None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:25.882459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 746/1476] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:25.888475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5535804-8248-434d-863b-e4dc2b0de697 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:25.889010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5535804-8248-434d-863b-e4dc2b0de697 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:25.889010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5535804-8248-434d-863b-e4dc2b0de697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:25.963723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d5535804-8248-434d-863b-e4dc2b0de697 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:25.964964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d5535804-8248-434d-863b-e4dc2b0de697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:26.070640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 731/1477] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:26.357474 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147fd2b-0749-4212-a9bb-6f2283caf4fe None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:26.358975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f147fd2b-0749-4212-a9bb-6f2283caf4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:26.513908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147fd2b-0749-4212-a9bb-6f2283caf4fe None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:26.587312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:26.635830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:26.636346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:26.636622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:26.636892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:26.637519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:26.639301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:05:26.639301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:05:26.639301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:05:26.640429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daca26e-25f6-4843-98e8-34b656c59a49 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:26.659530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 747/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Tue Aug 30 01:05:26 2022] GET /identity/v3/projects => generated 2553 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:05:28.608462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99f0cb-ba5c-438a-a689-d2b40e47b8a8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:28.609033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99f0cb-ba5c-438a-a689-d2b40e47b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:28.609033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d99f0cb-ba5c-438a-a689-d2b40e47b8a8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:28.610415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 732/1479] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:28.616585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea636da9-7ab3-4415-bc9b-54a9f0bb8c02 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:28.617017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea636da9-7ab3-4415-bc9b-54a9f0bb8c02 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:28.617017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea636da9-7ab3-4415-bc9b-54a9f0bb8c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:28.683216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ea636da9-7ab3-4415-bc9b-54a9f0bb8c02 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:28.684677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ea636da9-7ab3-4415-bc9b-54a9f0bb8c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:28.781262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 748/1480] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:29.076037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99f0cb-ba5c-438a-a689-d2b40e47b8a8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:29.077567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7d99f0cb-ba5c-438a-a689-d2b40e47b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:29.087760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:29.230541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d99f0cb-ba5c-438a-a689-d2b40e47b8a8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:29.313318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:29.345568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:29.346427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:29.346935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:29.347271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:29.349693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:29.443128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:29.444767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258610f6-2ef6-4dd7-bc7a-c456accdccd7 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:29.463453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 733/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:31.575840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9997-ce22-415c-b996-9c88201c8f3c None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:31.576432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9997-ce22-415c-b996-9c88201c8f3c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:31.576432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b9997-ce22-415c-b996-9c88201c8f3c None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:31.577723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 749/1482] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:31.583924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033e34-ed30-4a46-9d5a-0b09156dd4e4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:31.584391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033e34-ed30-4a46-9d5a-0b09156dd4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:31.584496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa033e34-ed30-4a46-9d5a-0b09156dd4e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:31.652763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aa033e34-ed30-4a46-9d5a-0b09156dd4e4 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:31.654027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa033e34-ed30-4a46-9d5a-0b09156dd4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:31.702473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:31.756875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 734/1483] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:32.048741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b9997-ce22-415c-b996-9c88201c8f3c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:32.050316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-575b9997-ce22-415c-b996-9c88201c8f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:32.074910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:32.156208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b9997-ce22-415c-b996-9c88201c8f3c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:32.230929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:32.231534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:32.231817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:32.232091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:32.233706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:32.309105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:32.344770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:32.346275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5000555-3e14-4d69-a526-ee731f6aee7c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:32.365227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 750/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:34.353493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b64831-e0d7-4ebe-9deb-459c5fe5fbda None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:34.354089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b64831-e0d7-4ebe-9deb-459c5fe5fbda None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:34.354089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b64831-e0d7-4ebe-9deb-459c5fe5fbda None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:34.355434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 735/1485] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:34.360896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1746768-3e6b-41cd-bfd9-eac72969dd9f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:34.361358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1746768-3e6b-41cd-bfd9-eac72969dd9f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:34.361471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1746768-3e6b-41cd-bfd9-eac72969dd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:34.430198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f1746768-3e6b-41cd-bfd9-eac72969dd9f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:34.431432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f1746768-3e6b-41cd-bfd9-eac72969dd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:34.532325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 751/1486] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:05:34.824273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b64831-e0d7-4ebe-9deb-459c5fe5fbda None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:34.829409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-53b64831-e0d7-4ebe-9deb-459c5fe5fbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:34.931741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b64831-e0d7-4ebe-9deb-459c5fe5fbda None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:35.006219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:35.008091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:35.008710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:35.008996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:35.009274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:35.010934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:35.123495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:35.124953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd776f9-b90d-4e8b-8e3f-3721c93a79e7 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:35.147096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 736/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:37.153059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ed5bb-d8f0-46d4-9874-4f0ba37f98c0 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:37.153626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ed5bb-d8f0-46d4-9874-4f0ba37f98c0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:37.153626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ed5bb-d8f0-46d4-9874-4f0ba37f98c0 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:37.154956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 752/1488] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:05:37.160180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216c32-9056-4dfb-8439-4f09355cbaa8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:37.160589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216c32-9056-4dfb-8439-4f09355cbaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:37.160741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5216c32-9056-4dfb-8439-4f09355cbaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:37.233549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e5216c32-9056-4dfb-8439-4f09355cbaa8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:37.234803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5216c32-9056-4dfb-8439-4f09355cbaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:37.294453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:37.340407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 737/1489] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:05:37.636702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ed5bb-d8f0-46d4-9874-4f0ba37f98c0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:37.638270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-877ed5bb-d8f0-46d4-9874-4f0ba37f98c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:37.675017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:37.744425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ed5bb-d8f0-46d4-9874-4f0ba37f98c0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:37.822143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:37.822737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:37.823013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:37.823290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:37.824882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:37.905670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:37.937011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:37.938522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec453c3-4c34-40c7-b9ce-ac2892b24129 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:37.957223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 753/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:40.460430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fad361-269c-4614-bd6e-b6f87f3154e9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:40.461035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fad361-269c-4614-bd6e-b6f87f3154e9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:40.461035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fad361-269c-4614-bd6e-b6f87f3154e9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:40.462373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 738/1491] 10.222.0.52 () {54 vars in 927 bytes} [Tue Aug 30 01:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:05:40.467659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f9ef-5cef-4651-9bb5-c2c564a70b58 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:40.468081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f9ef-5cef-4651-9bb5-c2c564a70b58 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:40.468192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4973f9ef-5cef-4651-9bb5-c2c564a70b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:40.535363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4973f9ef-5cef-4651-9bb5-c2c564a70b58 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:05:40.536663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4973f9ef-5cef-4651-9bb5-c2c564a70b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:40.654646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 754/1492] 10.222.0.52 () {58 vars in 1024 bytes} [Tue Aug 30 01:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:05:40.975376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fad361-269c-4614-bd6e-b6f87f3154e9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:05:40.976930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30fad361-269c-4614-bd6e-b6f87f3154e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:41.102104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fad361-269c-4614-bd6e-b6f87f3154e9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:05:41.179977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:05:41.180582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:05:41.180862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:05:41.181124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:05:41.182770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:05:41.192896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:41.297717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:05:41.303526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d3480f-a066-401a-a477-f4431491248e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:05:41.319219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 739/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:05:50.652852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=82 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:05:51.316363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=50 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:06.739826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:06.740431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:06.740431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:06.816901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:06.818153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:06.922510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 755/1494] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:06.931956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:06.932474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:06.932583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:07.010895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:07.012029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:07.113946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 740/1495] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:07.118688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:07.119912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:07.131426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966b4a55-b714-4f6f-9a09-84a8f34ea7f0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:07.196096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:07.230129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:07.230738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:07.231019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:07.231292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:07.231931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:07:07.232231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:07.233258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:07.233258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:07:07.233258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:07:07.234172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:07.245983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 756/1496] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:07:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:07:07.254522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:07.263947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e9b53-3037-4a48-b564-175c458cbcde None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:07.326359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:07.361297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:07.361934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:07.362300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:07.362533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:07.363250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:07.365642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:07.365642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:07:07.365642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:07:07.367164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:07.533104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 741/1497] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:07:07 2022] POST /identity/v3/projects => generated 379 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:07.537554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:07.539123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:07.573346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb9e8e9-f979-4511-9581-406cc9d462f6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:07.668589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:07.669245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:07.669529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:07.669831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:07.670458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:07.671484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:07.671484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:07:07.671484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:07:07.672747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:07.796340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 757/1498] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:07:07 2022] POST /identity/v3/users => generated 290 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:07:07.797571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:07.799176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:07.866477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c725f5-dafd-401e-8651-7875878e86cd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:07.960337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:07.970833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:07.971454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:07.971806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:07.972263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:07.972959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:07.974017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:07.974017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:07:07.974017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:07:07.974912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:07.988155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 742/1499] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:07:07.992345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:07.993982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:08.006100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc66efe-5b7b-4033-960f-d49cbac60cfc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:08.031291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:08.101909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:08.102431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:08.102712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:08.102979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:08.103613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:08.104777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:08.104777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:07:08.104777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:07:08.105660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db70648-1a5e-4323-b1f6-2a53464763d7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:08.118673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 758/1500] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:07:08.122994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:08.124552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:08.158386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8906a1fa-b776-4566-aae0-2c8abd548fbd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:08.260712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:08.261376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:08.261657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:08.261965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] PATH_INFO: `/v3/projects/4820807e725f4501a6a7210fc7020b2d/users/4fdb3a0e32574c9186d986779a8cf391/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:08.285192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4820807e725f4501a6a7210fc7020b2d, user_id=4fdb3a0e32574c9186d986779a8cf391, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:08.286271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:08.286271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:07:08.286271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:07:08.288235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49b4a-7954-421e-b811-6fc87e3ed4de admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:08.381689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 743/1501] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:07:08 2022] PUT /identity/v3/projects/4820807e725f4501a6a7210fc7020b2d/users/4fdb3a0e32574c9186d986779a8cf391/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:08.394354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dec60-7819-4217-ba42-ec4f671de590 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:08.394637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dec60-7819-4217-ba42-ec4f671de590 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:08.394912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dec60-7819-4217-ba42-ec4f671de590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:08.452103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-898dec60-7819-4217-ba42-ec4f671de590 None None] MFA Rules not processed for user `4fdb3a0e32574c9186d986779a8cf391`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:08.453302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-898dec60-7819-4217-ba42-ec4f671de590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:08.532957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 759/1502] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:07:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:08.546994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d069248-35b2-4bd7-ab91-4f79af18c4a1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:08.547289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d069248-35b2-4bd7-ab91-4f79af18c4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:08.547572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d069248-35b2-4bd7-ab91-4f79af18c4a1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:08.549132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 744/1503] 10.222.0.52 () {54 vars in 964 bytes} [Tue Aug 30 01:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:07:08.554218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0f11b-579f-4d3c-a8ea-dffe11aceb44 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:08.554510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0f11b-579f-4d3c-a8ea-dffe11aceb44 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:08.554786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb0f11b-579f-4d3c-a8ea-dffe11aceb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:08.587634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:08.631044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-aeb0f11b-579f-4d3c-a8ea-dffe11aceb44 None None] MFA Rules not processed for user `fb22d91ffd224b6aa9093a440d5e4925`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:08.632302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-aeb0f11b-579f-4d3c-a8ea-dffe11aceb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:08.708969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 760/1504] 10.222.0.52 () {58 vars in 1061 bytes} [Tue Aug 30 01:07:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:08.716769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d069248-35b2-4bd7-ab91-4f79af18c4a1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:08.718343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d069248-35b2-4bd7-ab91-4f79af18c4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:08.770850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:08.824875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d069248-35b2-4bd7-ab91-4f79af18c4a1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:08.901556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:08.902112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:08.902399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:08.902680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:08.904286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:09.004870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:09.006394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:09.009990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:09.024868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 745/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:09.070774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aed9a-500f-44d7-9054-2e37fa277048 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:09.071302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aed9a-500f-44d7-9054-2e37fa277048 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:09.071394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649aed9a-500f-44d7-9054-2e37fa277048 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:09.072581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 761/1506] 10.222.0.52 () {52 vars in 830 bytes} [Tue Aug 30 01:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:07:09.076295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:09.077556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:09.178802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360fdc1-df6f-48c2-86b8-6629f5e5ff91 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:09.279744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:09.280379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:09.280664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:09.280937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] PATH_INFO: `/v3/users/4fdb3a0e32574c9186d986779a8cf391` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:09.293788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fdb3a0e32574c9186d986779a8cf391)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:09.294826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:09.294826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:07:09.294826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:07:09.296039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:09.307336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:09.559621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fdb3a0e32574c9186d986779a8cf391'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:09.575813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fdb3a0e32574c9186d986779a8cf391'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:09.718572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fdb3a0e32574c9186d986779a8cf391'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:09.719024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fdb3a0e32574c9186d986779a8cf391'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:09.744694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fdb3a0e32574c9186d986779a8cf391'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:09.750344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 746/1507] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:07:09 2022] DELETE /identity/v3/users/4fdb3a0e32574c9186d986779a8cf391 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:09.754984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:09.755277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:09.755557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:09.813021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:07:09.814263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:09.892434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:09.905253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 762/1508] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:07:09.915551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:09.916818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:10.021340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8f7e2-065d-4b0a-aec3-accbd36f70fb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:10.077926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:10.098717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:10.102356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:10.102356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:10.102356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:10.102356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:10.216494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:10.218085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c5a95d-caa2-4a2b-a041-4a28f669600b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:10.238829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 747/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:07:10.749091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:07:10.750356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:07:10.764702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380655fc-15e2-4cd7-add8-48364e0a9fae None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:07:10.868493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:07:10.869129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:07:10.869409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:07:10.869686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] PATH_INFO: `/v3/projects/4820807e725f4501a6a7210fc7020b2d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:07:10.883040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4820807e725f4501a6a7210fc7020b2d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:07:10.884921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:07:10.884921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:07:10.884921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:07:10.886480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:07:10.994405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4820807e725f4501a6a7210fc7020b2d', 'request_id': 'req-7054f438-09d1-435e-aae3-a5bf54647358'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:11.115570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4820807e725f4501a6a7210fc7020b2d', 'request_id': 'req-7054f438-09d1-435e-aae3-a5bf54647358'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:11.131149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4820807e725f4501a6a7210fc7020b2d', 'request_id': 'req-7054f438-09d1-435e-aae3-a5bf54647358'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:11.154330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] The token cache is being invalidate because project 4820807e725f4501a6a7210fc7020b2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:07:11.154624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:11.155063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7054f438-09d1-435e-aae3-a5bf54647358 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:07:11.156481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 763/1510] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:07:10 2022] DELETE /identity/v3/projects/4820807e725f4501a6a7210fc7020b2d => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:07:20.302341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=62 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:07:21.152236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:41.132878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c9e6c-e49a-4b22-910a-7e6ef11a3b45 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:41.133611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c9e6c-e49a-4b22-910a-7e6ef11a3b45 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:41.133611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35c9e6c-e49a-4b22-910a-7e6ef11a3b45 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:41.134867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:08:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:41.139677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c246c73-b4df-4e0d-8b08-4c5fdc666332 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:41.140252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c246c73-b4df-4e0d-8b08-4c5fdc666332 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:41.140252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c246c73-b4df-4e0d-8b08-4c5fdc666332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:41.222596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5c246c73-b4df-4e0d-8b08-4c5fdc666332 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:41.223806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5c246c73-b4df-4e0d-8b08-4c5fdc666332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:41.328602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:41.483691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c9e6c-e49a-4b22-910a-7e6ef11a3b45 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:41.486608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d35c9e6c-e49a-4b22-910a-7e6ef11a3b45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:41.610045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35c9e6c-e49a-4b22-910a-7e6ef11a3b45 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:41.616290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a0497-1485-46cb-b8b0-79767790331a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:41.616897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a0497-1485-46cb-b8b0-79767790331a service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:41.617175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a0497-1485-46cb-b8b0-79767790331a service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:41.617450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a0497-1485-46cb-b8b0-79767790331a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:41.625686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a0497-1485-46cb-b8b0-79767790331a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:41.627879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9a0497-1485-46cb-b8b0-79767790331a service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:41.637084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 749/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:42.473777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f981d2-5cbf-448c-8094-7682502de7b4 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:42.474338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f981d2-5cbf-448c-8094-7682502de7b4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:42.474338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f981d2-5cbf-448c-8094-7682502de7b4 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:42.475688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:42.481624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276d4b7-8d97-4055-a505-607a941bcea3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:42.482199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276d4b7-8d97-4055-a505-607a941bcea3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:42.482199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c276d4b7-8d97-4055-a505-607a941bcea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:42.509177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c276d4b7-8d97-4055-a505-607a941bcea3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:42.510509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c276d4b7-8d97-4055-a505-607a941bcea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:42.520345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:08:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:42.674901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f981d2-5cbf-448c-8094-7682502de7b4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:42.689874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f981d2-5cbf-448c-8094-7682502de7b4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:42.695324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9cae1-66df-4dac-b851-014c2c3860e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:42.695923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9cae1-66df-4dac-b851-014c2c3860e2 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:42.696203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9cae1-66df-4dac-b851-014c2c3860e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:42.696481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9cae1-66df-4dac-b851-014c2c3860e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:42.705655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9cae1-66df-4dac-b851-014c2c3860e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:42.707375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9cae1-66df-4dac-b851-014c2c3860e2 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:42.716951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 766/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:43.185374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2f84-e36a-4983-9e83-76f974fedb3d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:43.185666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2f84-e36a-4983-9e83-76f974fedb3d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:43.185981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f2f84-e36a-4983-9e83-76f974fedb3d None None] PATH_INFO: `/v3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:43.187267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 751/1517] 10.222.0.74 () {54 vars in 925 bytes} [Tue Aug 30 01:08:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:08:43.194214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ccfef-f442-4ff9-ab23-f7808b5fbc97 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:43.194508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ccfef-f442-4ff9-ab23-f7808b5fbc97 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:43.194790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ccfef-f442-4ff9-ab23-f7808b5fbc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:43.254298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d31ccfef-f442-4ff9-ab23-f7808b5fbc97 None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:43.255507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d31ccfef-f442-4ff9-ab23-f7808b5fbc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:43.272020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:43.327017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 767/1518] 10.222.0.74 () {58 vars in 1015 bytes} [Tue Aug 30 01:08:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:52.498857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=56 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:08:53.130345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ceb3-6c9d-4fa1-96f4-854cf58f0338 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.130660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ceb3-6c9d-4fa1-96f4-854cf58f0338 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.131026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ceb3-6c9d-4fa1-96f4-854cf58f0338 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.132463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 752/1519] 10.222.0.52 () {54 vars in 961 bytes} [Tue Aug 30 01:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:53.137955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8e664-bc54-4a52-9168-b8fcd25fd28d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.138496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8e664-bc54-4a52-9168-b8fcd25fd28d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.138496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc8e664-bc54-4a52-9168-b8fcd25fd28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.158810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6dc8e664-bc54-4a52-9168-b8fcd25fd28d None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:53.160047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6dc8e664-bc54-4a52-9168-b8fcd25fd28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:53.169263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 768/1520] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Aug 30 01:08:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:53.177560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ceb3-6c9d-4fa1-96f4-854cf58f0338 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:53.191837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ceb3-6c9d-4fa1-96f4-854cf58f0338 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:53.196846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e9acd-ebad-4256-bdfa-3343fcbf0b5d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:53.197480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9acd-ebad-4256-bdfa-3343fcbf0b5d service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:53.197787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9acd-ebad-4256-bdfa-3343fcbf0b5d service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:53.198063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e9acd-ebad-4256-bdfa-3343fcbf0b5d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:53.208780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e9acd-ebad-4256-bdfa-3343fcbf0b5d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:53.211066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e9acd-ebad-4256-bdfa-3343fcbf0b5d service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:53.224313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 753/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:08:54.000768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad1854-f396-43f5-b391-8a2f7639461b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:54.001192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad1854-f396-43f5-b391-8a2f7639461b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:54.001598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad1854-f396-43f5-b391-8a2f7639461b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:54.003094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:54.008378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356def41-6594-4da7-a2ea-bf681cfbecc2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:54.008875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356def41-6594-4da7-a2ea-bf681cfbecc2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:54.008997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356def41-6594-4da7-a2ea-bf681cfbecc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:54.029792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-356def41-6594-4da7-a2ea-bf681cfbecc2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:54.031013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-356def41-6594-4da7-a2ea-bf681cfbecc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:54.040189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:54.197433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad1854-f396-43f5-b391-8a2f7639461b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:54.212541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad1854-f396-43f5-b391-8a2f7639461b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:54.218067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f53b96-d924-44e5-8411-e95776fd0768 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:54.218657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f53b96-d924-44e5-8411-e95776fd0768 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:54.218940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f53b96-d924-44e5-8411-e95776fd0768 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:54.219304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f53b96-d924-44e5-8411-e95776fd0768 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:54.228272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f53b96-d924-44e5-8411-e95776fd0768 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:54.229916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f53b96-d924-44e5-8411-e95776fd0768 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:54.239456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 770/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:55.082593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb79a0-80c3-40e8-8d0c-b4b78500d386 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.083141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb79a0-80c3-40e8-8d0c-b4b78500d386 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.083141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb79a0-80c3-40e8-8d0c-b4b78500d386 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.084489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 01:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:55.089687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518562-8468-4b95-b1ca-f2286321edda None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.090005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518562-8468-4b95-b1ca-f2286321edda None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.090315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71518562-8468-4b95-b1ca-f2286321edda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.112048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-71518562-8468-4b95-b1ca-f2286321edda None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:55.113269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-71518562-8468-4b95-b1ca-f2286321edda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:55.121243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 01:08:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:55.277128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb79a0-80c3-40e8-8d0c-b4b78500d386 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:55.292119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb79a0-80c3-40e8-8d0c-b4b78500d386 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:55.297511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d731fe9-a452-4cc7-b191-33d4368608e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:55.298281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731fe9-a452-4cc7-b191-33d4368608e1 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:55.298614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731fe9-a452-4cc7-b191-33d4368608e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:55.298943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d731fe9-a452-4cc7-b191-33d4368608e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:55.308678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731fe9-a452-4cc7-b191-33d4368608e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:55.310849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d731fe9-a452-4cc7-b191-33d4368608e1 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:55.321260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 756/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:08:59.173929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed38e5-434b-4dcc-8ba5-be3fc91af984 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:59.174503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed38e5-434b-4dcc-8ba5-be3fc91af984 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:59.174611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed38e5-434b-4dcc-8ba5-be3fc91af984 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:59.175931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:08:59.181885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbc2f-3ba4-4008-ba59-c66ebae4b57e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:59.185279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbc2f-3ba4-4008-ba59-c66ebae4b57e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:59.185279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbbc2f-3ba4-4008-ba59-c66ebae4b57e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:59.203379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b7cbbc2f-3ba4-4008-ba59-c66ebae4b57e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:08:59.204660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b7cbbc2f-3ba4-4008-ba59-c66ebae4b57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:59.212662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:08:59.515670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed38e5-434b-4dcc-8ba5-be3fc91af984 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:08:59.517915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3bed38e5-434b-4dcc-8ba5-be3fc91af984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:08:59.601546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed38e5-434b-4dcc-8ba5-be3fc91af984 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:08:59.605225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967588c4-419a-4a6c-ad9c-bb286480af6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:08:59.605939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967588c4-419a-4a6c-ad9c-bb286480af6f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:08:59.606269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967588c4-419a-4a6c-ad9c-bb286480af6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:08:59.606536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967588c4-419a-4a6c-ad9c-bb286480af6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:08:59.617379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967588c4-419a-4a6c-ad9c-bb286480af6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:08:59.618946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967588c4-419a-4a6c-ad9c-bb286480af6f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:08:59.628676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 773/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:08:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:01.530826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7ced-22a2-4a99-bc71-5914d4c41279 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:01.531386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7ced-22a2-4a99-bc71-5914d4c41279 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:01.531386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382d7ced-22a2-4a99-bc71-5914d4c41279 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:01.533271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:01.538801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8f4d0-ed9e-48b8-b8ef-641de4ee66fd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:01.539243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8f4d0-ed9e-48b8-b8ef-641de4ee66fd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:01.539441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8f4d0-ed9e-48b8-b8ef-641de4ee66fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:01.560050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f6c8f4d0-ed9e-48b8-b8ef-641de4ee66fd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:01.561237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f6c8f4d0-ed9e-48b8-b8ef-641de4ee66fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:01.570175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:01.856806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7ced-22a2-4a99-bc71-5914d4c41279 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:01.867833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382d7ced-22a2-4a99-bc71-5914d4c41279 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:01.871393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d8ce4-e98f-434c-8b0d-54a8a207e9b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:01.871984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d8ce4-e98f-434c-8b0d-54a8a207e9b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:01.872267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d8ce4-e98f-434c-8b0d-54a8a207e9b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:01.872545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d8ce4-e98f-434c-8b0d-54a8a207e9b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:01.882075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d8ce4-e98f-434c-8b0d-54a8a207e9b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:01.883494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186d8ce4-e98f-434c-8b0d-54a8a207e9b0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:01.893824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 759/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:05.451052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa34084-163b-4d8a-8865-837c2818969b None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.451697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa34084-163b-4d8a-8865-837c2818969b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.451697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa34084-163b-4d8a-8865-837c2818969b None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.453047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:05.458860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f81c5-ded6-41a0-86da-feb311344f65 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:05.459297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f81c5-ded6-41a0-86da-feb311344f65 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:05.459493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7f81c5-ded6-41a0-86da-feb311344f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:05.480559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ce7f81c5-ded6-41a0-86da-feb311344f65 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:05.481823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ce7f81c5-ded6-41a0-86da-feb311344f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:05.492253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:09:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:06.053129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa34084-163b-4d8a-8865-837c2818969b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:06.054860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7aa34084-163b-4d8a-8865-837c2818969b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:06.121010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:06.134356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa34084-163b-4d8a-8865-837c2818969b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:06.138056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddc700-de39-4501-9f84-d228dd9b803f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:06.138661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddc700-de39-4501-9f84-d228dd9b803f service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:06.138943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddc700-de39-4501-9f84-d228dd9b803f service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:06.139219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddc700-de39-4501-9f84-d228dd9b803f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:06.149154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddc700-de39-4501-9f84-d228dd9b803f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:06.150646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddc700-de39-4501-9f84-d228dd9b803f service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:06.160883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 776/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:08.130578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21745c-968b-4bf4-851d-5b9944ee0f5b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.131144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21745c-968b-4bf4-851d-5b9944ee0f5b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.131144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b21745c-968b-4bf4-851d-5b9944ee0f5b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.132479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 01:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:09:08.137648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c82e3f-5ee3-4621-ab4f-887113bd1f29 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.138130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c82e3f-5ee3-4621-ab4f-887113bd1f29 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.138238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c82e3f-5ee3-4621-ab4f-887113bd1f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.158218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c9c82e3f-5ee3-4621-ab4f-887113bd1f29 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:09:08.159436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c9c82e3f-5ee3-4621-ab4f-887113bd1f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:08.167216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 01:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:09:08.717310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21745c-968b-4bf4-851d-5b9944ee0f5b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:09:08.719056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b21745c-968b-4bf4-851d-5b9944ee0f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:09:08.732441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b21745c-968b-4bf4-851d-5b9944ee0f5b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:09:08.736817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10015e97-1f70-42ff-a486-860cdc15569e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:09:08.737416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015e97-1f70-42ff-a486-860cdc15569e service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:09:08.737700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015e97-1f70-42ff-a486-860cdc15569e service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:09:08.737999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015e97-1f70-42ff-a486-860cdc15569e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:09:08.747397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10015e97-1f70-42ff-a486-860cdc15569e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:09:08.748754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10015e97-1f70-42ff-a486-860cdc15569e service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:09:08.759603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 762/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:09:18.153296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=11 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:09:18.755359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=33 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:34.608320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d217146-eb88-47b4-9d18-326eaf8e7761 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.609613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d217146-eb88-47b4-9d18-326eaf8e7761 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.609613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d217146-eb88-47b4-9d18-326eaf8e7761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.635827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7d217146-eb88-47b4-9d18-326eaf8e7761 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.637064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7d217146-eb88-47b4-9d18-326eaf8e7761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.650262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 778/1540] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.661986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.662573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.662573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.689100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b840e-4e91-49bf-98f3-cf35fa87e7bc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.689343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b840e-4e91-49bf-98f3-cf35fa87e7bc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.689631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704b840e-4e91-49bf-98f3-cf35fa87e7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.698404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.699736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.729851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 763/1541] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.732763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-704b840e-4e91-49bf-98f3-cf35fa87e7bc None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.732763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-704b840e-4e91-49bf-98f3-cf35fa87e7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.736187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:34.755819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7777f82-83cb-4e69-a9a4-d11ec9b2126e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:34.756021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 779/1542] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.756021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.756021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.756267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.763986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:34.764629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.766027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.766317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.771304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:34.771755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:34.772819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:10:34.772819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:10:34.772819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:10:34.773787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3176f85-3ff0-49f6-bea6-efd4efbd0bfd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:34.800678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.802003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.804544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 764/1543] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:10:34.805908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e5ed-8195-4975-9312-2db04b396e05 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.806195 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e5ed-8195-4975-9312-2db04b396e05 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.806477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d98e5ed-8195-4975-9312-2db04b396e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.820671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 780/1544] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.821520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:34.837555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6c49b7-b845-4679-87e0-47407d8b95d5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:34.847384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5d98e5ed-8195-4975-9312-2db04b396e05 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.849133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:34.849780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.850069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.850341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.850987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:34.852337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5d98e5ed-8195-4975-9312-2db04b396e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.853831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:10:34.853831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:10:34.853831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:10:34.854987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7be4b-4f31-4d65-9c0e-80c89e1ddb91 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:34.872415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 765/1545] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.873922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296b1a1-e543-4a4d-ae5e-67c6a2b00297 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.874874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296b1a1-e543-4a4d-ae5e-67c6a2b00297 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.875161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296b1a1-e543-4a4d-ae5e-67c6a2b00297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.908680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c296b1a1-e543-4a4d-ae5e-67c6a2b00297 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.909938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c296b1a1-e543-4a4d-ae5e-67c6a2b00297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.918059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 766/1546] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.919337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.920250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.920537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.943174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:34.945005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:34.953397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 767/1547] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:34.954059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:34.965334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64a25e-ce31-47a4-a36a-e907b4eab699 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:34.970186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:34.970803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.971088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.971366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.972020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:34.972326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:34.973951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e576166-6096-4046-9a6b-d759a08d4255 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:34.978819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 781/1548] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/projects => generated 393 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:34.981209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd26f0e-de54-492f-af69-133bc22bd751 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.981209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd26f0e-de54-492f-af69-133bc22bd751 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.981209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd26f0e-de54-492f-af69-133bc22bd751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:34.993716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 768/1549] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:34.993716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:34.993716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:34.993716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.009565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3cd26f0e-de54-492f-af69-133bc22bd751 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.010730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3cd26f0e-de54-492f-af69-133bc22bd751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.019110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.020446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.080282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 782/1550] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.080282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0406c4-4e85-4c9f-bcf3-a4e7f693d215 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.080282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0406c4-4e85-4c9f-bcf3-a4e7f693d215 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.080282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0406c4-4e85-4c9f-bcf3-a4e7f693d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.087371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 769/1551] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.088331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.098002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f6a4d-2b80-49b0-a4d8-cf238fe79577 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.099578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cb0406c4-4e85-4c9f-bcf3-a4e7f693d215 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.100770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cb0406c4-4e85-4c9f-bcf3-a4e7f693d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.102963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.103645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.104032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.104405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.105125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:35.105601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.107302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.109122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 783/1552] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.110444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.110790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.111130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.120172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 770/1553] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:35.120988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.132887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2cccb-8927-488b-943d-388efa0de594 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.135046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.136292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.137666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.138308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.138592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.138864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.139488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.141315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:10:35.141315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:10:35.141315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:10:35.142558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef13819d-3640-44dc-8d40-33b9a19e303a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.145208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 784/1554] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.145856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.155899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803bd82-7466-489c-9197-ebe5ab32cf48 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.160666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c522272-7bc9-44aa-84b3-599ce1a3452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.161271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522272-7bc9-44aa-84b3-599ce1a3452f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.161557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522272-7bc9-44aa-84b3-599ce1a3452f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.161876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c522272-7bc9-44aa-84b3-599ce1a3452f admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.162511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522272-7bc9-44aa-84b3-599ce1a3452f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.164466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c522272-7bc9-44aa-84b3-599ce1a3452f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.262402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 771/1555] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/users => generated 297 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:35.263706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc57ffa-95e5-4c3d-9856-787bca05aae1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.264064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc57ffa-95e5-4c3d-9856-787bca05aae1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.264342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc57ffa-95e5-4c3d-9856-787bca05aae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.284198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4cc57ffa-95e5-4c3d-9856-787bca05aae1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.285382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4cc57ffa-95e5-4c3d-9856-787bca05aae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.293276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 772/1556] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.294471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.307196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.307196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.307412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 785/1557] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:35.307412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.307412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.307412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.314933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.316130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.328744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.329907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.395146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 773/1558] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.396270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.398894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 786/1559] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.399599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.408107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20030ff0-d3f2-4e04-a5ea-790673a33425 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.409385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c114cea-f091-4d31-ac49-71893ee22153 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.412946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.413807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.413951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.414170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.414769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.414901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:35.415109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.415380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.415662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.415942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.416585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.416706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff93eec-6768-4e29-8ec8-016ef09d9f72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.418518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.425135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:35.429901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 774/1560] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:35.431124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.431409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.431692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.440965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:35.452818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.454224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.462191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 775/1561] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.466015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.473127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f5dee2-aa85-4e85-97c5-0bd858fb6ff1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.479032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215bba1-5a40-4cb2-8179-1706064ebeda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.479158 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215bba1-5a40-4cb2-8179-1706064ebeda admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.479360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215bba1-5a40-4cb2-8179-1706064ebeda admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.479640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215bba1-5a40-4cb2-8179-1706064ebeda admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.480299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215bba1-5a40-4cb2-8179-1706064ebeda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.481924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215bba1-5a40-4cb2-8179-1706064ebeda admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.493421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 776/1562] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:35.495068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.495333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.495609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.520272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:35.520272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:35.524946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 777/1563] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:35.525701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.535002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01a8b5-74a1-4384-9cc6-a86e454d96a3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.539627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.540219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.546593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.546593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.546593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.546593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.662624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 787/1564] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/projects => generated 373 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:35.663475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.736822 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a95ce19-5bbc-454e-b01b-5a04ff6e80ea admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.741475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.742107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.742395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.742657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.743297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:35.743593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.745234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.757777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 788/1565] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:35.758763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.770415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc481b0-4439-4a04-a306-f1acba915ad5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.775118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.775717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.776008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.776268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.776899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:35.777224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.778846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.792064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 789/1566] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:35.792882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.803389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50209302-3b22-4df2-a9b8-c6ef68479be0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.808046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.808634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.808910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.809181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.809835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.811932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.886396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 778/1567] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/users => generated 289 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:35.886396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.895361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6624fd1e-3f2b-4c3b-a344-d7a90294e999 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.899891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.900476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.900757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.901033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.901672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:35.902013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.903592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.914688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 790/1568] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] POST /identity/v3/projects => generated 383 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:35.915650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.920310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 779/1569] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:35.928076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:35.937218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999a7a93-bca9-4151-910f-0daa4c91f591 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.987759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74b418-4c54-4618-9a4c-43492134ac69 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:35.992660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.993271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.993556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.993873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.994010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:35.994538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.994669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:35.994921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:35.995196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:35.995833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:35.996164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:35.996273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:35.997794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.007355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 791/1570] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:36.008239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.010266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 780/1571] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:36.011249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.019670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5f3cf-b374-4867-9a95-9fcfdf4e21f1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.024723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.025367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.025651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.025970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.026643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.027852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2dddc-5a62-4a56-b5d2-65913478f9aa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.028591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.032751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.033422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.033705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.039544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.039544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.039544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.158319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 792/1572] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.159125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.168676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196cc6f4-7257-4e8c-9cca-1fe738f2884c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.174213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.175063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.175346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.175624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.176252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.180918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 781/1573] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/projects => generated 363 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.181099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.181211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.242332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab4614-c2ec-4e52-86ce-30fc7adb192f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.246940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.247551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.247838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.248119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.248804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.252225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.266265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 782/1574] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:36.267426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.277430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b07eb-667e-4e18-ab26-1db5bf080b3c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.283585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.284344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.284679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.284923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.285693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.287729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.292136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 793/1575] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/projects => generated 389 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.296004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.298755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:36.303793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:36.355896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83dc3a-1b35-4afa-8060-2d4a92e7028e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.360839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.361807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.361807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.362026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.362707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.364667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.385902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 783/1576] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.387280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.403700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba0ab6d-9aed-42a6-994e-3ac6e07d6078 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.409626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.410377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.410663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.410946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] PATH_INFO: `/v3/projects/9f0605a8a2fe48e49a410127b5f3b32c/users/76ea3ed3bb2b412093004c7e0f49c7ce/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.429195 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0605a8a2fe48e49a410127b5f3b32c, user_id=76ea3ed3bb2b412093004c7e0f49c7ce, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.432200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.483235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 794/1577] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/projects => generated 377 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.483235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.529723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 784/1578] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:36 2022] PUT /identity/v3/projects/9f0605a8a2fe48e49a410127b5f3b32c/users/76ea3ed3bb2b412093004c7e0f49c7ce/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:36.534946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.542296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf022e-99d8-403d-8894-ed9b13175667 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.598853 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcaba3-b15c-4ee8-822f-7c75983544e9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.600658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.601282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.601567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.601908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.602582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.603758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.604373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.604509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.604661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.604941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.605613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.607230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.619281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 785/1579] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:36.620176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.630132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f276c-8869-4034-a747-25b28f8e90e6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.634729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.635323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.635606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.635881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.636501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.638478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.743702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 795/1580] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/projects => generated 367 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.744719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.754146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 786/1581] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/users => generated 282 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.755443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.765635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759025a7-080b-4513-b1aa-12d17da8fbf7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.813011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c74cca-d7d6-435c-ba5a-4efd53f1e572 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.818057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.818779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.819099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.819380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.820043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.822794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.825510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:36.826067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f2f60-f618-4723-8f77-b29ca7aa81b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.826685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f2f60-f618-4723-8f77-b29ca7aa81b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.826965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f2f60-f618-4723-8f77-b29ca7aa81b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.827366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f2f60-f618-4723-8f77-b29ca7aa81b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.828078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f2f60-f618-4723-8f77-b29ca7aa81b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.830022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f2f60-f618-4723-8f77-b29ca7aa81b9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.835163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 796/1582] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:36.835865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.846729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ac689-5866-43a5-99b1-0cc8f1e4b353 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.851432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.852039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.852325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.852627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.853264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.854942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.865657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 797/1583] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:36.866627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.877026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4738e63-691e-4a81-9f51-303b8fd37574 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.881648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.882286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.882573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.882852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.883476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.885382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.929979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 787/1584] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/users => generated 295 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:36.932123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.932411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.932691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.953068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:36.954315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:36.962225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 788/1585] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:36.963008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:36.972326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d1121a-f946-4b01-88a5-9d16364e048d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:36.977218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:36.977857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:36.978143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:36.978421 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:36.979092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:36.980724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:36.991890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 789/1586] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:36.992684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.003234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac14087-52be-4cdb-b685-28a003e2259b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.007883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.020862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.020862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.020862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.020862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.020862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.066977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 798/1587] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/users => generated 289 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:37.067845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.078007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266a56b-36e8-460f-8a5e-ea79bd44f77c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.082728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.083321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.083604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.083889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.084524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.086179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.097063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 799/1588] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.101042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.107712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e8b1f-b531-4220-9054-5a6df4cc553d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.112870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.113497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.113810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.114089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] PATH_INFO: `/v3/projects/13862ab3a9504644853a8c48c3f69b4c/users/90486e175bb2411b9f2adedf6e3ecac6/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.129621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=13862ab3a9504644853a8c48c3f69b4c, user_id=90486e175bb2411b9f2adedf6e3ecac6, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.130700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:10:37.130700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:10:37.130700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:10:37.132617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.164218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 790/1589] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:36 2022] POST /identity/v3/users => generated 284 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:37.165147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.170051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=11,keystone:SELECT=89 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:37.176199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a69203-dcd3-47ab-9f4e-a68624e6d6d8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.180839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.181440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.181741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.182028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.182657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.184244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.195586 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 791/1590] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.196472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.206087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6622fe-1720-4fac-9699-c43da9d242e8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.210739 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.211331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.211612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.211887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.212517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.214131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.215701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 800/1591] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:37 2022] PUT /identity/v3/projects/13862ab3a9504644853a8c48c3f69b4c/users/90486e175bb2411b9f2adedf6e3ecac6/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:37.216572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.225642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 792/1592] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.226890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.238062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5743a8a0-23dc-4f08-b536-04e9658cb577 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.273599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e361298-6f8e-4e14-b598-8f76bbced2bb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.277395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.278041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.278324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.278630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.295850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.300464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.300464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.300464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] PATH_INFO: `/v3/projects/9aaae18da02e4446a8898dc57ee662ec/users/88d31346e1104a2a878cc87e30d68fd6/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.314750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aaae18da02e4446a8898dc57ee662ec, user_id=88d31346e1104a2a878cc87e30d68fd6, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.318280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.340655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.342138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.352448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 801/1593] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:37.353339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.363240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56bbdb9-19ae-4983-95f0-8bf7f0ff624d service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.367948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.368549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.368829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.369118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.369751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.371349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.374243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:37.382496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 802/1594] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.386600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.393178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c75ee-00c4-4367-b5dc-805460f8f5bd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.397721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.398342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.398644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.398897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.399520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.401100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.411397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 803/1595] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.412360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.422811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 793/1596] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:37 2022] PUT /identity/v3/projects/9aaae18da02e4446a8898dc57ee662ec/users/88d31346e1104a2a878cc87e30d68fd6/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:37.423375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15a3f75-b385-49a1-b68f-5624bd9edca0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.423821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.484680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9876229-12ae-4403-ab77-a081bcb8629f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.485362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876229-12ae-4403-ab77-a081bcb8629f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.485652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876229-12ae-4403-ab77-a081bcb8629f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.485990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9876229-12ae-4403-ab77-a081bcb8629f admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.486761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876229-12ae-4403-ab77-a081bcb8629f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.488672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9876229-12ae-4403-ab77-a081bcb8629f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.498952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 804/1597] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.500446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.500739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.501149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.504882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89b13c3-5d2b-4b38-8e4f-697da44f11a6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.510387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29e89c-835b-4bd5-a46d-e5769b24e358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.511175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29e89c-835b-4bd5-a46d-e5769b24e358 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.511460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29e89c-835b-4bd5-a46d-e5769b24e358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.511834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29e89c-835b-4bd5-a46d-e5769b24e358 admin admin] PATH_INFO: `/v3/projects/11c8f48af7394a75ab6013ca066751a7/users/942d175e7e2742728e66fea84c851038/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.521074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:37.522401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:37.527621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29e89c-835b-4bd5-a46d-e5769b24e358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c8f48af7394a75ab6013ca066751a7, user_id=942d175e7e2742728e66fea84c851038, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.530889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29e89c-835b-4bd5-a46d-e5769b24e358 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.531594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 805/1598] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:37.532235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.542805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f1a50-a58f-43ab-ac78-0154c740a06d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.548685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.552767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.552767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.552767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.552767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.552767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.563891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 806/1599] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.564957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.575486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b64dda-3cf9-43a6-b5e1-6fbde0292c0e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.581524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.587540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.587540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.587540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.587540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.587540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.596483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 807/1600] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.597353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.609272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b39d4-0e76-4223-9547-88b60909bf51 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.616120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 794/1601] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:37 2022] PUT /identity/v3/projects/11c8f48af7394a75ab6013ca066751a7/users/942d175e7e2742728e66fea84c851038/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:37.617393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.624266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.624266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.649909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] MFA Rules not processed for user `88d31346e1104a2a878cc87e30d68fd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:37.651142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:37.669086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.669796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.670107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.670385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] PATH_INFO: `/v3/projects/63fc23ed59f3478a8ebf9891df53bd96/users/26292168591d4073b3b093396dbbc4dd/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.684951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63fc23ed59f3478a8ebf9891df53bd96, user_id=26292168591d4073b3b093396dbbc4dd, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.685684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:37.687967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.718820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 795/1602] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:37.720083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.733756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc32d84-83ca-4aaa-b804-86910b079f6d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.739389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.739932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.740303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.740483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.741134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.742798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.756887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 796/1603] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:37.765318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.765318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:37.788917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 808/1604] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:37 2022] PUT /identity/v3/projects/63fc23ed59f3478a8ebf9891df53bd96/users/26292168591d4073b3b093396dbbc4dd/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:37.798894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.842058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bf9ef-54a1-4f96-8267-338deb7dd785 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.847901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:37.871558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50ec794-6989-47b8-b3cb-65e94cea16d1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.876802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205cb981-5bb4-4288-919b-a12a856a43de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.877457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cb981-5bb4-4288-919b-a12a856a43de admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.877767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cb981-5bb4-4288-919b-a12a856a43de admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.878033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cb981-5bb4-4288-919b-a12a856a43de admin admin] PATH_INFO: `/v3/projects/742c089b0dea4b1f89efd31f48f9ffc5/users/2ffe5c7cb44f476e913d64003105002c/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.892411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cb981-5bb4-4288-919b-a12a856a43de admin admin] RBAC: Authorizing `identity:create_grant(project_id=742c089b0dea4b1f89efd31f48f9ffc5, user_id=2ffe5c7cb44f476e913d64003105002c, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.893795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.893795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.893795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.893795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.895107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205cb981-5bb4-4288-919b-a12a856a43de admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.903437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.904874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.916546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 797/1605] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:37.916546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:37.926802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1bf93f-1ddd-4363-a9db-b0d0ab7e9f35 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:37.932018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04ee6b-e572-49e4-bffb-a7b6fe19b0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:37.932805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04ee6b-e572-49e4-bffb-a7b6fe19b0f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.933112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04ee6b-e572-49e4-bffb-a7b6fe19b0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.933392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04ee6b-e572-49e4-bffb-a7b6fe19b0f7 admin admin] PATH_INFO: `/v3/projects/4f082e13ad014501b7a781167ddb2ed8/users/13848c47ddd9441282bec982cccc8b91/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:37.950727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04ee6b-e572-49e4-bffb-a7b6fe19b0f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f082e13ad014501b7a781167ddb2ed8, user_id=13848c47ddd9441282bec982cccc8b91, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:37.953841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04ee6b-e572-49e4-bffb-a7b6fe19b0f7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:37.981311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 809/1606] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:37 2022] PUT /identity/v3/projects/742c089b0dea4b1f89efd31f48f9ffc5/users/2ffe5c7cb44f476e913d64003105002c/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:37.982961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83eee9-26ba-4ef8-8efd-cfdc514c940c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:37.983914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83eee9-26ba-4ef8-8efd-cfdc514c940c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:37.984208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de83eee9-26ba-4ef8-8efd-cfdc514c940c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.003584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-de83eee9-26ba-4ef8-8efd-cfdc514c940c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:38.005462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-de83eee9-26ba-4ef8-8efd-cfdc514c940c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.070965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 810/1607] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.072275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38903-6de6-4c16-9832-5190981e632a None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.072572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38903-6de6-4c16-9832-5190981e632a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.072850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a38903-6de6-4c16-9832-5190981e632a None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.074936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 811/1608] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 30 01:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:38.075607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38903-6de6-4c16-9832-5190981e632a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.092490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a38903-6de6-4c16-9832-5190981e632a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.104291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 798/1609] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:37 2022] PUT /identity/v3/projects/4f082e13ad014501b7a781167ddb2ed8/users/13848c47ddd9441282bec982cccc8b91/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:38.105961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209e14-38b9-454a-b037-c9e4d308b34e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.106665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209e14-38b9-454a-b037-c9e4d308b34e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.107160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c209e14-38b9-454a-b037-c9e4d308b34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.113065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aa6b9b-666f-4471-9bfd-4988e46f67e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.114747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6b9b-666f-4471-9bfd-4988e46f67e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.115045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6b9b-666f-4471-9bfd-4988e46f67e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.115500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aa6b9b-666f-4471-9bfd-4988e46f67e7 admin admin] PATH_INFO: `/v3/projects/bece3987711840c1b9b0c3cd3b575da1/users/ff6d6c114e964b9eb6a85d14733f6382/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.136530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa6b9b-666f-4471-9bfd-4988e46f67e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bece3987711840c1b9b0c3cd3b575da1, user_id=ff6d6c114e964b9eb6a85d14733f6382, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.140494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2aa6b9b-666f-4471-9bfd-4988e46f67e7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.152728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8c209e14-38b9-454a-b037-c9e4d308b34e None None] MFA Rules not processed for user `26292168591d4073b3b093396dbbc4dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:38.154790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8c209e14-38b9-454a-b037-c9e4d308b34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.205378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:38.223863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 799/1610] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.225355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d24192-7685-4347-97b1-7369be7212ef None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.225584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d24192-7685-4347-97b1-7369be7212ef None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.225918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d24192-7685-4347-97b1-7369be7212ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.246424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-14d24192-7685-4347-97b1-7369be7212ef None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:38.247637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-14d24192-7685-4347-97b1-7369be7212ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.324126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 812/1611] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:38 2022] PUT /identity/v3/projects/bece3987711840c1b9b0c3cd3b575da1/users/ff6d6c114e964b9eb6a85d14733f6382/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:38.324126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dec50-8bb0-4eac-895f-7f9631f976a2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.324126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dec50-8bb0-4eac-895f-7f9631f976a2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.324126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dec50-8bb0-4eac-895f-7f9631f976a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.329067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 800/1612] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.330637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d24192-7685-4347-97b1-7369be7212ef None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.362811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ee2dec50-8bb0-4eac-895f-7f9631f976a2 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:38.364289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ee2dec50-8bb0-4eac-895f-7f9631f976a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.403883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d24192-7685-4347-97b1-7369be7212ef None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.437977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 813/1613] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.439324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.439623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.439947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.461520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:38.462804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.466225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f0773-3175-471a-84d8-4b6c15546fc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.466950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f0773-3175-471a-84d8-4b6c15546fc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.467245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f0773-3175-471a-84d8-4b6c15546fc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.467527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f0773-3175-471a-84d8-4b6c15546fc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.478493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:38.538797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 814/1614] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.539577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.542141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f0773-3175-471a-84d8-4b6c15546fc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.543767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f0773-3175-471a-84d8-4b6c15546fc5 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.550425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f97642-d310-4b29-a219-e3bdeda4d9d5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.555173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 801/1615] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:38.555477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.556145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.556398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.556905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.557022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.557524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.557642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.558220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.560171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.562848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:38.593413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] MFA Rules not processed for user `ff6d6c114e964b9eb6a85d14733f6382`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:38.594673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:38.643249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 815/1616] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:38.643249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.672540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 802/1617] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:38.675439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.689757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3383a3b-8c2e-4249-a48b-7a1b83b2b145 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.711288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907dd72-848a-475e-9c2c-94b764e53a6f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.715239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.716018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.716357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.716897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.770711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.771370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.771650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.771901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.811200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.814107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.834376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 816/1618] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:38.837147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.839741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.841588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.853922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce531d8a-4e66-4646-948d-92d6c3f0f525 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.856902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 803/1619] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:38.858074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.860990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.862290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.862810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.863321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.875271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501a723-bbcc-4059-8f02-55754ee268c1 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.880886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.882399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.883344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.883843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.883966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.884573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.885646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.888364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:38.904355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 817/1620] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:38.911904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:38.919231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e0fd13-3c9a-4407-9f41-7ccd02291c66 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:38.927756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:38.928480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:38.928815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:38.929609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:38.930458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:38.933001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.000187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 804/1621] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/users => generated 289 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:39.004670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.011516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:39.022067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeac30-56df-496f-ab5c-0c8395e67e01 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.025593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 818/1622] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:38 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:39.027238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.027888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.028280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.028572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.029219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.030042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.031784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.045463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:39.048560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 805/1623] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:10:39.049829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.060142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb601c-bed0-4d21-972a-8f9b0b2b47ff admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.136233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f71d2e-95d3-46d4-8ed9-012365a80a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.136998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f71d2e-95d3-46d4-8ed9-012365a80a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.137284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f71d2e-95d3-46d4-8ed9-012365a80a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.137567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f71d2e-95d3-46d4-8ed9-012365a80a00 admin admin] PATH_INFO: `/v3/projects/a100dd0ca0a7413f9a79a3abd432533c/users/4525516a47d34402b656837c6f0f6062/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.160075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f71d2e-95d3-46d4-8ed9-012365a80a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a100dd0ca0a7413f9a79a3abd432533c, user_id=4525516a47d34402b656837c6f0f6062, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.161172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f71d2e-95d3-46d4-8ed9-012365a80a00 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.217271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ac0d3-4273-4acd-adc0-dce85ac86ec7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.222623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f953731-1db3-4c83-906c-c0848b93fcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.223412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f953731-1db3-4c83-906c-c0848b93fcef admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.223693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f953731-1db3-4c83-906c-c0848b93fcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.223966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f953731-1db3-4c83-906c-c0848b93fcef admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.224610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f953731-1db3-4c83-906c-c0848b93fcef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.226675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f953731-1db3-4c83-906c-c0848b93fcef admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.278416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 806/1624] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:39 2022] PUT /identity/v3/projects/a100dd0ca0a7413f9a79a3abd432533c/users/4525516a47d34402b656837c6f0f6062/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:39.280212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd383-d6e6-402e-a4b9-38928fb6f6db None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.280932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd383-d6e6-402e-a4b9-38928fb6f6db None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.281374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd383-d6e6-402e-a4b9-38928fb6f6db None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.285462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 807/1625] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 30 01:10:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:39.287574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bd383-d6e6-402e-a4b9-38928fb6f6db None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.369623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 819/1626] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:39 2022] POST /identity/v3/users => generated 291 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:39.369623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a455e39-1540-4ebc-8a32-aab7e183aedc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.369623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a455e39-1540-4ebc-8a32-aab7e183aedc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.369623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a455e39-1540-4ebc-8a32-aab7e183aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.395222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bd383-d6e6-402e-a4b9-38928fb6f6db None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.400984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.401629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.402000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.402285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.402943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.404745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.420304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7a455e39-1540-4ebc-8a32-aab7e183aedc None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:39.420715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 808/1627] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:39.422596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.422896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7a455e39-1540-4ebc-8a32-aab7e183aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:39.446660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe7386-c61d-40e3-9484-37280a543ffe admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.457667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.458330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.458980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.458980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.459528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.469487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.489146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 809/1628] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:39.490148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.503139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414cb7f2-7e89-43d3-93b6-08973d7292d8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.509683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.510392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.511093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.511384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] PATH_INFO: `/v3/projects/a100dd0ca0a7413f9a79a3abd432533c/users/4525516a47d34402b656837c6f0f6062/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.514436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a100dd0ca0a7413f9a79a3abd432533c, user_id=4525516a47d34402b656837c6f0f6062, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.518991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.533511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 810/1629] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:39 2022] PUT /identity/v3/projects/a100dd0ca0a7413f9a79a3abd432533c/users/4525516a47d34402b656837c6f0f6062/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:39.534477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.543048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 820/1630] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:10:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:39.544826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.544948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.545247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.600750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] MFA Rules not processed for user `4525516a47d34402b656837c6f0f6062`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:39.603454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:39.643890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:39.663831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7519183-6b0d-43a8-8123-8629ae7d2876 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:39.686047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:39.687735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:39.688107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:39.688619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] PATH_INFO: `/v3/projects/f43afb9a96834b7598a7399ae4bfbfc0/users/8a96662f96264ab29741d8d9a7716d29/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:39.730407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43afb9a96834b7598a7399ae4bfbfc0, user_id=8a96662f96264ab29741d8d9a7716d29, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:39.738993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:39.751404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 821/1631] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:39.752665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.806398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:39.859206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 811/1632] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:39 2022] PUT /identity/v3/projects/f43afb9a96834b7598a7399ae4bfbfc0/users/8a96662f96264ab29741d8d9a7716d29/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:39.862277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:39.878535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c8b4d-5205-4bea-a73e-8be306aea042 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:40.386351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:40.386351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.386351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.386351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.394437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8163e-3114-4b09-b616-012d8c18ad4c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:40.394437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:40.394437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.394437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.397907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.399047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:40.399047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:40.401870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:40.401870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:40.402447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 822/1633] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:40.402447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:40.403155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 812/1634] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:40.403449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101a2e5a-324d-4528-b67d-8823b7338cca service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:40.403449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a24dc-8c17-4018-8e95-83ff3f866f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:40.403449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a24dc-8c17-4018-8e95-83ff3f866f21 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.403449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a24dc-8c17-4018-8e95-83ff3f866f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.404140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a24dc-8c17-4018-8e95-83ff3f866f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.404140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a24dc-8c17-4018-8e95-83ff3f866f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:40.404140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a24dc-8c17-4018-8e95-83ff3f866f21 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:40.404140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 823/1635] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:40.404668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:40.404668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc1c68-cefc-4960-82d2-b1acd1c62c97 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:40.404668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:40.404668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.405082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.405082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] PATH_INFO: `/v3/projects/f43afb9a96834b7598a7399ae4bfbfc0/users/8a96662f96264ab29741d8d9a7716d29/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.405082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f43afb9a96834b7598a7399ae4bfbfc0, user_id=8a96662f96264ab29741d8d9a7716d29, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:40.405082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:40.406143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.406143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.406143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.413246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 813/1636] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:40 2022] PUT /identity/v3/projects/f43afb9a96834b7598a7399ae4bfbfc0/users/8a96662f96264ab29741d8d9a7716d29/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:40.413393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] MFA Rules not processed for user `8a96662f96264ab29741d8d9a7716d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:40.413393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:40.549494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 824/1637] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:10:40.768669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:40.904371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f3a53-7f0c-4225-8472-aa09cf7e7df6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:40.908931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e139abc-ea70-4a9f-b9cf-6fb292e8b504 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:40.909792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e139abc-ea70-4a9f-b9cf-6fb292e8b504 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:40.910087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e139abc-ea70-4a9f-b9cf-6fb292e8b504 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:40.910362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e139abc-ea70-4a9f-b9cf-6fb292e8b504 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:40.925021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e139abc-ea70-4a9f-b9cf-6fb292e8b504 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:40.932451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e139abc-ea70-4a9f-b9cf-6fb292e8b504 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:40.976012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 814/1638] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:41.189982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:41.201575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c92e2c-2072-44fc-ab63-b900b91819b2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:41.206840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce92b8d-aaea-458d-b3e3-3262e6f60b52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:41.207534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92b8d-aaea-458d-b3e3-3262e6f60b52 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:41.207820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92b8d-aaea-458d-b3e3-3262e6f60b52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:41.208101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce92b8d-aaea-458d-b3e3-3262e6f60b52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:41.278611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92b8d-aaea-458d-b3e3-3262e6f60b52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:41.281763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce92b8d-aaea-458d-b3e3-3262e6f60b52 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:41.299582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 825/1639] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:10:41.301298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:42.350215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b84da-a9a5-4121-a943-e879cf7bdf1d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:42.350215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b84da-a9a5-4121-a943-e879cf7bdf1d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:42.350215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13b84da-a9a5-4121-a943-e879cf7bdf1d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:42.350215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 815/1640] 10.222.0.52 () {54 vars in 962 bytes} [Tue Aug 30 01:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:42.370639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51b02f-507b-4df7-8ca3-6865b84e8b83 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:42.371908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51b02f-507b-4df7-8ca3-6865b84e8b83 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:42.372636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f51b02f-507b-4df7-8ca3-6865b84e8b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:42.412410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1f51b02f-507b-4df7-8ca3-6865b84e8b83 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:42.414658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1f51b02f-507b-4df7-8ca3-6865b84e8b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:42.540501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 826/1641] 10.222.0.52 () {58 vars in 1059 bytes} [Tue Aug 30 01:10:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:42.549635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d582dc-8873-4d8a-9050-feb7e5a3776b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:42.549635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d582dc-8873-4d8a-9050-feb7e5a3776b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:42.549635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d582dc-8873-4d8a-9050-feb7e5a3776b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:42.550379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 816/1642] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 30 01:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:42.557390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:42.557939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:42.558389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:42.584663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:42.586404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:42.601006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 827/1643] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:10:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:42.623261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d582dc-8873-4d8a-9050-feb7e5a3776b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:42.642804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d582dc-8873-4d8a-9050-feb7e5a3776b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:42.647401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bde0f04-87fa-49b1-b30f-a31cc8165618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:42.649527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0f04-87fa-49b1-b30f-a31cc8165618 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:42.650323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0f04-87fa-49b1-b30f-a31cc8165618 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:42.650789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bde0f04-87fa-49b1-b30f-a31cc8165618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:42.665893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0f04-87fa-49b1-b30f-a31cc8165618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:42.668292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bde0f04-87fa-49b1-b30f-a31cc8165618 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:42.680163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:42.725115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 817/1644] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:46.106326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:46.148523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:46.275105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38b348-dbeb-4177-ab6c-6350ff73a489 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:46.291623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:46.292321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:46.292643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:46.293264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] PATH_INFO: `/v3/users/88d31346e1104a2a878cc87e30d68fd6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:46.304221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88d31346e1104a2a878cc87e30d68fd6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:46.304699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: WARNING py.warnings [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:10:46.304699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: warnings.warn(msg) Aug 30 01:10:46.304699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: Aug 30 01:10:46.305628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:46.800923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88d31346e1104a2a878cc87e30d68fd6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:46.839524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88d31346e1104a2a878cc87e30d68fd6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:46.842826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88d31346e1104a2a878cc87e30d68fd6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:46.844506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88d31346e1104a2a878cc87e30d68fd6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:46.897818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88d31346e1104a2a878cc87e30d68fd6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:47.018492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 828/1645] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:10:46 2022] DELETE /identity/v3/users/88d31346e1104a2a878cc87e30d68fd6 => generated 0 bytes in 912 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:47.021808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:47.022397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:47.022584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:47.083873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:47.090459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:47.404132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 818/1646] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:47.446082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:47.447876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:47.626172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:47.627618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:47.644903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a1c4c-8310-4913-9df7-5c8f403ccbc3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:47.657868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a4bb9-81c5-435f-b62f-f684985cd9bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:47.661115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a4bb9-81c5-435f-b62f-f684985cd9bb service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:47.661451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a4bb9-81c5-435f-b62f-f684985cd9bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:47.661886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a4bb9-81c5-435f-b62f-f684985cd9bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:47.662413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf17488-79a2-4520-9d06-2903af70cfc8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:47.685460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:47.690352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:47.690726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:47.691016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] PATH_INFO: `/v3/users/26292168591d4073b3b093396dbbc4dd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:47.696675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26292168591d4073b3b093396dbbc4dd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:47.713445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:47.713637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a4bb9-81c5-435f-b62f-f684985cd9bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:47.715054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a4bb9-81c5-435f-b62f-f684985cd9bb service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:47.757868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 829/1647] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:48.044657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26292168591d4073b3b093396dbbc4dd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:48.070096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26292168591d4073b3b093396dbbc4dd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:48.187754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26292168591d4073b3b093396dbbc4dd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:48.188864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26292168591d4073b3b093396dbbc4dd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:48.229404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26292168591d4073b3b093396dbbc4dd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:48.251755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 819/1648] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:10:47 2022] DELETE /identity/v3/users/26292168591d4073b3b093396dbbc4dd => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:10:48.253289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:48.253702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:48.253923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:48.317544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:48.319392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:48.398676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:48.400217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:48.454955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:48.547847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 830/1649] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:48.612636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df3508-cc35-449e-97f9-c4421882b931 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:48.631960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:48.631960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:48.634153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:48.635088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:48.637591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:48.639218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] PATH_INFO: `/v3/users/ff6d6c114e964b9eb6a85d14733f6382` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:48.644038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff6d6c114e964b9eb6a85d14733f6382)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:48.657904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:48.706062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:48.783843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703c6fa-6996-48d7-a588-0d7d1d0c0be4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:48.803261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcff42f-8695-481e-901e-16e521c4850a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:48.803910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcff42f-8695-481e-901e-16e521c4850a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:48.804288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcff42f-8695-481e-901e-16e521c4850a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:48.804688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcff42f-8695-481e-901e-16e521c4850a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:48.824973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcff42f-8695-481e-901e-16e521c4850a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:48.826549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcff42f-8695-481e-901e-16e521c4850a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:48.850566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 831/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:48.975387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff6d6c114e964b9eb6a85d14733f6382'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:48.990109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff6d6c114e964b9eb6a85d14733f6382'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:49.112504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff6d6c114e964b9eb6a85d14733f6382'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:49.114634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff6d6c114e964b9eb6a85d14733f6382'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:49.141772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff6d6c114e964b9eb6a85d14733f6382'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:49.154548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 820/1651] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:10:48 2022] DELETE /identity/v3/users/ff6d6c114e964b9eb6a85d14733f6382 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:49.156007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:49.156427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:49.156714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:49.191156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:49.192367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:49.346073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 832/1652] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:49.449405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:49.451261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:49.583141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f6169-2031-4423-8966-66749e34fa2c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:49.587164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:49.587807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:49.588155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:49.588435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:49.607062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:49.610203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:49.637385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 821/1653] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:50.558214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:50.563248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:50.574299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:50.577784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:50.585381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b575cd8-3288-4c3e-9c79-6ed6ef40918f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:50.591391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:50.592042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:50.592413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:50.593629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] PATH_INFO: `/v3/projects/63fc23ed59f3478a8ebf9891df53bd96` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:50.596001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63fc23ed59f3478a8ebf9891df53bd96)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:50.598813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:50.600934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d6917b-f167-45b6-8812-dda975b088c0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:50.617424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:50.618352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:50.618809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:50.619406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] PATH_INFO: `/v3/projects/9aaae18da02e4446a8898dc57ee662ec` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:50.620949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aaae18da02e4446a8898dc57ee662ec)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:50.622438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 01:10:50.622438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 30 01:10:50.622438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: Aug 30 01:10:50.624204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:50.827563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63fc23ed59f3478a8ebf9891df53bd96', 'request_id': 'req-1b4ad9fc-446c-4052-a703-f5a7980754e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.864808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aaae18da02e4446a8898dc57ee662ec', 'request_id': 'req-69cc9596-27aa-4451-89e9-e376ffa1ba72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.893183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aaae18da02e4446a8898dc57ee662ec', 'request_id': 'req-69cc9596-27aa-4451-89e9-e376ffa1ba72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.912206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aaae18da02e4446a8898dc57ee662ec', 'request_id': 'req-69cc9596-27aa-4451-89e9-e376ffa1ba72'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.948351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63fc23ed59f3478a8ebf9891df53bd96', 'request_id': 'req-1b4ad9fc-446c-4052-a703-f5a7980754e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.960639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63fc23ed59f3478a8ebf9891df53bd96', 'request_id': 'req-1b4ad9fc-446c-4052-a703-f5a7980754e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.991129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] The token cache is being invalidate because project 63fc23ed59f3478a8ebf9891df53bd96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:10:50.991418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.993444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:50.995852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 833/1654] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:10:50 2022] DELETE /identity/v3/projects/63fc23ed59f3478a8ebf9891df53bd96 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:51.000570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:51.003748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:51.104194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] The token cache is being invalidate because project 9aaae18da02e4446a8898dc57ee662ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:10:51.105129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.107841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-69cc9596-27aa-4451-89e9-e376ffa1ba72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.111431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 822/1655] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:10:50 2022] DELETE /identity/v3/projects/9aaae18da02e4446a8898dc57ee662ec => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:51.116374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795bf3f-9887-486c-b346-27ad3910622a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:51.117315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795bf3f-9887-486c-b346-27ad3910622a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:51.117864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1795bf3f-9887-486c-b346-27ad3910622a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:51.172469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1795bf3f-9887-486c-b346-27ad3910622a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:51.173050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ad9fc-446c-4052-a703-f5a7980754e5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:51.175209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1795bf3f-9887-486c-b346-27ad3910622a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:51.323424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:51.324079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:51.324360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:51.324664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] PATH_INFO: `/v3/users/4525516a47d34402b656837c6f0f6062` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:51.326140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4525516a47d34402b656837c6f0f6062)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:51.331814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:51.337453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 823/1656] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:51.345165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691d99-36af-4d71-927c-caa852167562 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:51.345536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691d99-36af-4d71-927c-caa852167562 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:51.345977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd691d99-36af-4d71-927c-caa852167562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:51.400750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cd691d99-36af-4d71-927c-caa852167562 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:51.404878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd691d99-36af-4d71-927c-caa852167562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:51.419422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 824/1657] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:51.425631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691d99-36af-4d71-927c-caa852167562 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:51.444410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd691d99-36af-4d71-927c-caa852167562 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:51.466131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:51.466772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:51.467054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:51.469335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:51.470160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:51.470477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:51.475177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:51.496050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 825/1658] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:10:51.501361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:51.515695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0a088-d4e0-4f13-b95b-408bac43b661 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:51.525845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:51.527125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:51.527125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:51.527804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:51.528642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:51.532589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:51.551983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:51.690473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4525516a47d34402b656837c6f0f6062'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.698557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4525516a47d34402b656837c6f0f6062'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.702331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4525516a47d34402b656837c6f0f6062'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.704648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4525516a47d34402b656837c6f0f6062'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.720249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 826/1659] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:51 2022] POST /identity/v3/projects => generated 357 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:51.726526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:51.731274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:51.745378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e009172d-eba3-46fc-aaa1-18686defc328 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4525516a47d34402b656837c6f0f6062'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:51.878119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ad006-3435-4134-b5f4-9d53ac2756fb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:51.890697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:51.891240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:51.891527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:51.891893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:51.892517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:51.894585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:51.908263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:51.915030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 834/1660] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:10:50 2022] DELETE /identity/v3/users/4525516a47d34402b656837c6f0f6062 => generated 0 bytes in 915 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:52.026470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.027238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.027652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.047592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 827/1661] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:51 2022] POST /identity/v3/users => generated 279 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:52.053178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:52.105716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5c0af-f5fa-481b-9ad6-aa7b3aa196e2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:52.111660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] MFA Rules not processed for user `90486e175bb2411b9f2adedf6e3ecac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:52.112922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:52.263183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 835/1662] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:52.297132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:52.297132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:52.462627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:52.463957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.464246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.464526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.469562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:52.472090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:52.508105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 828/1663] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:52.514083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:52.522065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:52.530570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128f039c-50d6-4465-aeac-82576a095cf3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:52.551851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:52.552801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.552801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.553342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.557995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd1080-8b02-4d20-b1bb-ff452da12987 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:52.573774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:52.574475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.574760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.575040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] PATH_INFO: `/v3/projects/bece3987711840c1b9b0c3cd3b575da1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.576330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bece3987711840c1b9b0c3cd3b575da1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:52.580936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:52.584600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:52.586484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:52.612036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 836/1664] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:52.616581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:52.645349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9efd3-97e6-4e8e-870d-a9b0e2e4e2d2 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:52.671342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:52.672246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.672634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.673020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.673812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:52.675954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:52.713744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 837/1665] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:52.717857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:52.728794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a9c42-ef4c-4f04-bc59-e88f676d436b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:52.752145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33e998-cce9-42f9-bc03-b6e3b85089b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:52.753081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e998-cce9-42f9-bc03-b6e3b85089b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.753587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e998-cce9-42f9-bc03-b6e3b85089b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.754189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33e998-cce9-42f9-bc03-b6e3b85089b7 admin admin] PATH_INFO: `/v3/projects/cd483df7bb9242ae8364aaa85c24b9f3/users/ca8478dec48d46db9ff19e715b60078e/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.785979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33e998-cce9-42f9-bc03-b6e3b85089b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd483df7bb9242ae8364aaa85c24b9f3, user_id=ca8478dec48d46db9ff19e715b60078e, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:52.789386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33e998-cce9-42f9-bc03-b6e3b85089b7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:52.811891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bece3987711840c1b9b0c3cd3b575da1', 'request_id': 'req-04d21497-f8fc-40e6-b1e7-b554222317b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:52.838114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bece3987711840c1b9b0c3cd3b575da1', 'request_id': 'req-04d21497-f8fc-40e6-b1e7-b554222317b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:52.866891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bece3987711840c1b9b0c3cd3b575da1', 'request_id': 'req-04d21497-f8fc-40e6-b1e7-b554222317b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:52.897032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 838/1666] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:52 2022] PUT /identity/v3/projects/cd483df7bb9242ae8364aaa85c24b9f3/users/ca8478dec48d46db9ff19e715b60078e/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:10:52.909147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:52.909551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:52.909986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:52.987440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] MFA Rules not processed for user `ca8478dec48d46db9ff19e715b60078e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:52.990223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:52.992248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] The token cache is being invalidate because project bece3987711840c1b9b0c3cd3b575da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:10:52.992553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:52.996590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:53.000958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 829/1667] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:10:52 2022] DELETE /identity/v3/projects/bece3987711840c1b9b0c3cd3b575da1 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:53.005114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:53.014352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:53.162557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d21497-f8fc-40e6-b1e7-b554222317b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:53.172803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 839/1668] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:10:53.181710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:53.196619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:53.340358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:53.343033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:53.347616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:53.349072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] PATH_INFO: `/v3/users/8a96662f96264ab29741d8d9a7716d29` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:53.356315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a96662f96264ab29741d8d9a7716d29)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:53.364131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:53.413424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74818df-94e1-4ab9-88e2-ca0f4a4474d4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:53.437355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:53.448368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:53.448368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:53.448368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:53.448368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:53.448368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:53.633827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 840/1669] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:53 2022] POST /identity/v3/projects => generated 357 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:10:53.637858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:53.653663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a96662f96264ab29741d8d9a7716d29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:53.667942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a96662f96264ab29741d8d9a7716d29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:53.758232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a96662f96264ab29741d8d9a7716d29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:53.758892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a96662f96264ab29741d8d9a7716d29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:53.772324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc9b3e-153f-45cf-98f2-5074a86c53c8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:53.783103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:53.783973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:53.784796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:53.785992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:53.787220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a96662f96264ab29741d8d9a7716d29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:53.788750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:53.796930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:53.807684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 830/1670] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:10:53 2022] DELETE /identity/v3/users/8a96662f96264ab29741d8d9a7716d29 => generated 0 bytes in 803 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:10:53.928117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 841/1671] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:53 2022] POST /identity/v3/users => generated 285 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:10:53.932482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:53.934231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:54.048537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e99371-db98-4c11-908f-27132e6fe491 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:54.057759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:54.059441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.059615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.060006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.060851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:54.063469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:54.090743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 831/1672] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:54.092753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:54.096709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:54.110838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f82699-53c7-45ae-b6f3-b2d195457891 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:54.126033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96de45c9-f9ac-438d-9ec0-1e19d92b6813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:54.127222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de45c9-f9ac-438d-9ec0-1e19d92b6813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.127222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de45c9-f9ac-438d-9ec0-1e19d92b6813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.128162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96de45c9-f9ac-438d-9ec0-1e19d92b6813 admin admin] PATH_INFO: `/v3/projects/f7e3694c8521423583aaaefdbe7d3bee/users/276d4fbe87d9459987048ffde9a5f36b/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.147482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:54.192943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de45c9-f9ac-438d-9ec0-1e19d92b6813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e3694c8521423583aaaefdbe7d3bee, user_id=276d4fbe87d9459987048ffde9a5f36b, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:54.199412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96de45c9-f9ac-438d-9ec0-1e19d92b6813 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:54.370351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 842/1673] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:54 2022] PUT /identity/v3/projects/f7e3694c8521423583aaaefdbe7d3bee/users/276d4fbe87d9459987048ffde9a5f36b/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:54.373284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:54.390833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e6dab3-ac11-4b13-bfcd-6faa4ef3cd28 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:54.581044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d11a29-7750-4d69-b1e7-e8d691aa3b99 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.581339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d11a29-7750-4d69-b1e7-e8d691aa3b99 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.581622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d11a29-7750-4d69-b1e7-e8d691aa3b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.607416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:54.608962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.609626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.609782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.611702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:54.621278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:54.668384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 832/1674] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:54.675830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:54.693703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-09d11a29-7750-4d69-b1e7-e8d691aa3b99 None None] MFA Rules not processed for user `76ea3ed3bb2b412093004c7e0f49c7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:54.697986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-09d11a29-7750-4d69-b1e7-e8d691aa3b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:54.720628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6928b-75cb-4770-9237-af54f209fba3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:54.743776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962c15b8-b415-483d-9487-15a1c28995f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:54.744652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c15b8-b415-483d-9487-15a1c28995f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.745069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c15b8-b415-483d-9487-15a1c28995f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.745353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962c15b8-b415-483d-9487-15a1c28995f5 admin admin] PATH_INFO: `/v3/projects/f7e3694c8521423583aaaefdbe7d3bee/users/276d4fbe87d9459987048ffde9a5f36b/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.750176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c15b8-b415-483d-9487-15a1c28995f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e3694c8521423583aaaefdbe7d3bee, user_id=276d4fbe87d9459987048ffde9a5f36b, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:54.753165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962c15b8-b415-483d-9487-15a1c28995f5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:54.799689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 833/1675] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:54 2022] PUT /identity/v3/projects/f7e3694c8521423583aaaefdbe7d3bee/users/276d4fbe87d9459987048ffde9a5f36b/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:54.800979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.801273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.801560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.834468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 843/1676] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:54.845492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:54.847470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:54.847781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:54.856507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] MFA Rules not processed for user `276d4fbe87d9459987048ffde9a5f36b`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:54.859359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:54.887322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] MFA Rules not processed for user `76ea3ed3bb2b412093004c7e0f49c7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:54.888575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:54.979407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 844/1677] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:10:54.983377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:54.983377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:55.021416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 834/1678] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:54 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:55.021416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:55.029235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:55.114026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b61b4-decc-4f64-8868-11c9aa14e02d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:55.132069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:55.133235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:55.139018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:55.139625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] PATH_INFO: `/v3/projects/a100dd0ca0a7413f9a79a3abd432533c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:55.142609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a100dd0ca0a7413f9a79a3abd432533c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:55.147577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:55.233995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a4a99-6aa3-4c4f-b56b-b486ccb8491a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:55.256343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:55.256343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:55.256343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:55.256343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:55.274934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:55.279131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:55.294436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 835/1679] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:55.297424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:55.299275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:55.390164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a100dd0ca0a7413f9a79a3abd432533c', 'request_id': 'req-c5092e9b-6ab6-4c3c-815d-250c89c2f823'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:55.420192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf85cb-e289-4b5f-ba31-f6d1739c049f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:55.599237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:55.600219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:55.600653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:55.601069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:55.603692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a100dd0ca0a7413f9a79a3abd432533c', 'request_id': 'req-c5092e9b-6ab6-4c3c-815d-250c89c2f823'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:55.614386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a100dd0ca0a7413f9a79a3abd432533c', 'request_id': 'req-c5092e9b-6ab6-4c3c-815d-250c89c2f823'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:55.666669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] The token cache is being invalidate because project a100dd0ca0a7413f9a79a3abd432533c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:10:55.666974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:55.677817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c5092e9b-6ab6-4c3c-815d-250c89c2f823 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:55.680279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 845/1680] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:10:54 2022] DELETE /identity/v3/projects/a100dd0ca0a7413f9a79a3abd432533c => generated 0 bytes in 700 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:55.698508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3019a6-5dd5-4647-800f-430d609dbd36 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:55.701221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3019a6-5dd5-4647-800f-430d609dbd36 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:55.701513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3019a6-5dd5-4647-800f-430d609dbd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:55.818397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4d3019a6-5dd5-4647-800f-430d609dbd36 None None] MFA Rules not processed for user `942d175e7e2742728e66fea84c851038`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:55.819844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4d3019a6-5dd5-4647-800f-430d609dbd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:55.893984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:55.896001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:55.930295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 836/1681] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 633 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:55.931408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:55.934394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:56.012803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:56.016793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 846/1682] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:56.053966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:56.053966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:56.053966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:56.098022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13defe0-c09d-413c-8529-2f0464a16567 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:56.103445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:56.105153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:56.106016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:56.106537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] PATH_INFO: `/v3/projects/f43afb9a96834b7598a7399ae4bfbfc0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:56.110457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f43afb9a96834b7598a7399ae4bfbfc0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:56.117918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:56.147524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:56.154893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:56.183538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 847/1683] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:56.200449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:56.202931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:56.348422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f43afb9a96834b7598a7399ae4bfbfc0', 'request_id': 'req-4064fc41-8668-42c9-8400-d612ee2fa532'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:56.376888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f43afb9a96834b7598a7399ae4bfbfc0', 'request_id': 'req-4064fc41-8668-42c9-8400-d612ee2fa532'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:56.412296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f43afb9a96834b7598a7399ae4bfbfc0', 'request_id': 'req-4064fc41-8668-42c9-8400-d612ee2fa532'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:56.516815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] The token cache is being invalidate because project f43afb9a96834b7598a7399ae4bfbfc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:10:56.519719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:56.519719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4064fc41-8668-42c9-8400-d612ee2fa532 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:10:56.523020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 837/1684] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:10:55 2022] DELETE /identity/v3/projects/f43afb9a96834b7598a7399ae4bfbfc0 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:56.524432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:56.527828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:56.528121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:56.537120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6a7c5d-b327-45c3-afc3-4925debeb832 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:56.626695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:56.627960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:56.637058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:56.773424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:56.774030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:56.774442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:56.777951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:56.789769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:56.856493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 838/1685] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:56.864832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:56.869669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:56.962761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:56.968052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54ae03-948b-40b5-ba15-ae8a8797299f service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:56.992268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 848/1686] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 808 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:57.000734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.000734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.000734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.013447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f937bb-d736-49f5-b860-5224acf2886a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.035212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.035716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.036013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.036357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.042627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:57.053614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:57.064253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:57.099493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 849/1687] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:57.100604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:57.131852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:57.158916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2255a2-509a-4c4f-91a0-6da0b8b9c6a9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.172818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.178810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.179264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.179664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.180478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:57.180906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.182757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d585aed-2b93-49c0-9f8a-003f61c3eb91 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:57.213921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 850/1688] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:10:57.218683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.219291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.220118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.221720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.223226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:57.249136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 839/1689] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:57.254135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:57.278192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15980763-5168-4862-b7d3-e54a14eb2bd5 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.293527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.294209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.294491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.294767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.295987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.303011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:57.307252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:57.309837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:57.384611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 851/1690] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:57.386097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:57.401413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36c57a-b069-4ba1-bead-ea0a9cf5e8c0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.416445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.417286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.417716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.422500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.422500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:10:57.422500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.422500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:57.461256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:57.479736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 852/1691] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:10:57.502940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:57.502940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0ca4c-6f4b-4bce-a050-98edea757c88 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.512827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a94998-4aed-406b-a3fa-8f135c853990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.513458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a94998-4aed-406b-a3fa-8f135c853990 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.513761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a94998-4aed-406b-a3fa-8f135c853990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.514490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a94998-4aed-406b-a3fa-8f135c853990 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.515062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a94998-4aed-406b-a3fa-8f135c853990 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.518921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a94998-4aed-406b-a3fa-8f135c853990 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:57.543667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 840/1692] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:57 2022] POST /identity/v3/projects => generated 385 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:57.547675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:57.691555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e13a-ada7-4cfd-9117-74c43250f115 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.700028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f247667-1d54-4789-bf7f-35b005c47081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.700959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247667-1d54-4789-bf7f-35b005c47081 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.701316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247667-1d54-4789-bf7f-35b005c47081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.702600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f247667-1d54-4789-bf7f-35b005c47081 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.704383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f247667-1d54-4789-bf7f-35b005c47081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.706968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f247667-1d54-4789-bf7f-35b005c47081 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:57.725197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 853/1693] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:57 2022] POST /identity/v3/projects => generated 377 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:57.729763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a944400-3cdf-4066-9e71-0a30382b47ff None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.730419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a944400-3cdf-4066-9e71-0a30382b47ff None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.730419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a944400-3cdf-4066-9e71-0a30382b47ff None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.743265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 854/1694] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 30 01:10:57 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:10:57.749268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a944400-3cdf-4066-9e71-0a30382b47ff None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:57.793842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 841/1695] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:57 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:57.795459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.795759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.796919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.905367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:57.909695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:57.929208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a944400-3cdf-4066-9e71-0a30382b47ff None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:57.942356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:57.943013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:57.943305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:57.943583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:57.946979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:57.950584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.041935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 842/1696] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:10:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:58.042628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.058352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e909a98-73a6-4e01-90a0-4bcf21391ff4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.066880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.068708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.069003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.069287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.070699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.077679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.082209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 855/1697] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:57 2022] POST /identity/v3/users => generated 289 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:58.086319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.114979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d094d9eb-cbe8-44f2-92ea-ecacecf8181b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.123076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 843/1698] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:58.127776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.138314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:58.142378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.143219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.143725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.148790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.148790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.148790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.156920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220ced4-21dc-4f18-aeda-bdda59170ddc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.161188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.162923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.163395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.163763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.176524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 856/1699] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:58.182036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.188568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.199358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.212337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45fa47b-68df-4c84-81e2-8139d29591d1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.219781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 844/1700] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:58.220787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.235371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.236099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.236418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.236699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.237988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.239706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.258806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e5f66-7cf7-422f-8ddc-455045f0d8dc service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.274212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 857/1701] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:58.278090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.281988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.282696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.283472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.285903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.287111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.288872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.309953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f93262-f248-4da4-a36b-44706a6cf307 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.319188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 845/1702] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:58.323858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.327290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4a051-59a4-49b9-9bd9-8ab0983efbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.328376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4a051-59a4-49b9-9bd9-8ab0983efbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.328767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4a051-59a4-49b9-9bd9-8ab0983efbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.329151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4a051-59a4-49b9-9bd9-8ab0983efbc2 admin admin] PATH_INFO: `/v3/projects/1064d44812324318a4805b8fe3c7f9c8/users/5d09f2a924f340bab640797a7bd61cdb/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.353835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27a8b6-7250-4887-8ca1-3f87bd3d225a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.355089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4a051-59a4-49b9-9bd9-8ab0983efbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1064d44812324318a4805b8fe3c7f9c8, user_id=5d09f2a924f340bab640797a7bd61cdb, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.361275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4a051-59a4-49b9-9bd9-8ab0983efbc2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.368957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d88b8-2a24-4615-b288-ade2d159d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.369690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d88b8-2a24-4615-b288-ade2d159d9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.370017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d88b8-2a24-4615-b288-ade2d159d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.370305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d88b8-2a24-4615-b288-ade2d159d9f2 admin admin] PATH_INFO: `/v3/projects/d7e8f24d91f94c698ae6018210deea1d/users/88c21dfd715c4befbb040ec9b74a15e5/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.398751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d88b8-2a24-4615-b288-ade2d159d9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e8f24d91f94c698ae6018210deea1d, user_id=88c21dfd715c4befbb040ec9b74a15e5, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.402751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d88b8-2a24-4615-b288-ade2d159d9f2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.435410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 858/1703] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:58 2022] PUT /identity/v3/projects/1064d44812324318a4805b8fe3c7f9c8/users/5d09f2a924f340bab640797a7bd61cdb/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:58.447982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.448319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.448558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.457650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:10:58.500058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 846/1704] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:10:58 2022] PUT /identity/v3/projects/d7e8f24d91f94c698ae6018210deea1d/users/88c21dfd715c4befbb040ec9b74a15e5/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:10:58.510188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4304-3a4d-4ae6-b36d-030fe810fb6a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.511296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4304-3a4d-4ae6-b36d-030fe810fb6a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.512147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f4304-3a4d-4ae6-b36d-030fe810fb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.550760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] MFA Rules not processed for user `5d09f2a924f340bab640797a7bd61cdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:58.552201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:58.591884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-059f4304-3a4d-4ae6-b36d-030fe810fb6a None None] MFA Rules not processed for user `88c21dfd715c4befbb040ec9b74a15e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:58.594069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-059f4304-3a4d-4ae6-b36d-030fe810fb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:58.693201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 859/1705] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:58.694563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:58.764910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 847/1706] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:58.767883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8bf44-98fd-483b-86c3-d532eb2cba20 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.771992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8bf44-98fd-483b-86c3-d532eb2cba20 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.771992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8bf44-98fd-483b-86c3-d532eb2cba20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.809434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b8d2b1-9087-482a-8588-4c5e90bb9dff None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:58.824559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:58.825518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:58.826012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:58.826688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:58.827892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:58.830093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:58.839962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3cc8bf44-98fd-483b-86c3-d532eb2cba20 None None] MFA Rules not processed for user `2ffe5c7cb44f476e913d64003105002c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:58.843535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3cc8bf44-98fd-483b-86c3-d532eb2cba20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:59.038794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 860/1707] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:58 2022] POST /identity/v3/projects => generated 387 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:59.039367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:59.117691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 848/1708] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:59.120395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.121403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.122803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.173084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] MFA Rules not processed for user `90486e175bb2411b9f2adedf6e3ecac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:59.175436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:59.206074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1368682-58e5-42c1-900c-6ea771a83a4b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:59.219145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346aae1-7144-47ce-a1d7-eca9d899debc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:59.220001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346aae1-7144-47ce-a1d7-eca9d899debc admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.220435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346aae1-7144-47ce-a1d7-eca9d899debc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.220862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346aae1-7144-47ce-a1d7-eca9d899debc admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.221517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346aae1-7144-47ce-a1d7-eca9d899debc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:59.223521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346aae1-7144-47ce-a1d7-eca9d899debc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:59.259248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 849/1709] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:59.259248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:59.274437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32693a-49aa-4ed3-a65a-f02cbfe0f26f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:59.279514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:59.280803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.281087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.281361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.282040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:59.283973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:59.334467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 861/1710] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:10:59 2022] POST /identity/v3/projects => generated 377 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:59.334467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.334467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.334467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.367932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] MFA Rules not processed for user `13848c47ddd9441282bec982cccc8b91`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:59.369196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:59.431142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 850/1711] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:59 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:59.432006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:59.433905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:59.521478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 862/1712] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:10:59.521478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:59.637831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bd660-4b10-4e5b-a871-9feeee400e83 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:59.662126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef71bc-855f-486c-8932-282ace5de675 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:59.671265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:59.672050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.672342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.672714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.673371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:59.675770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:59.736349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:59.737215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.737498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.737822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.807476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 863/1713] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:10:59 2022] POST /identity/v3/users => generated 295 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:10:59.818608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:59.832591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26830230-f210-4403-899d-c350081e24f6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:10:59.850963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c64b8-b9aa-424b-b488-478837037525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:10:59.850963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c64b8-b9aa-424b-b488-478837037525 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.850963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c64b8-b9aa-424b-b488-478837037525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.850963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c64b8-b9aa-424b-b488-478837037525 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.851773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c64b8-b9aa-424b-b488-478837037525 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:59.851773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c64b8-b9aa-424b-b488-478837037525 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:59.853317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:10:59.855264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:10:59.868699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 864/1714] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:10:59.869202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 851/1715] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:10:59.870231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:10:59.871802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:10:59.892864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:10:59.897947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:10:59.897947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:10:59.996153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] MFA Rules not processed for user `13848c47ddd9441282bec982cccc8b91`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:10:59.998095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:00.011907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 865/1716] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:00.013626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:00.033578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b443c-9f5d-42fe-9212-6d5e098f902b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.052156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.052791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.053310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.053435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.054053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:00.055686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:00.071185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6557453-4b8d-4e76-a4dd-085ef3b56732 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.071367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 866/1717] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:00.071367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:00.080019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.081221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.081596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca4080-ec7b-44bb-b5be-d68c7d005448 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.083182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.085470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.087555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.088885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:00.089011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.089367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.089773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.092793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:00.094854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:00.117172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:00.117450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 867/1718] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:00.117541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:00.117628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:00.129539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5a8ce6-7b0c-416e-bd11-72b135ea75ad admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.158363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 852/1719] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:10:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:00.159294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:00.159752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.164007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.164652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.165154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] PATH_INFO: `/v3/projects/ee9ba3eba02e493eb3058a1eb038fab9/users/2cd06e7233d74775abb4932266c8b268/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.175387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e24b2c-05a2-429d-b460-0e86973aa3d1 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.180512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.181539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.181881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.182210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] PATH_INFO: `/v3/projects/7bfd8735327f4caf93453e2207bf7c36/users/8d2b0408dc084ea6a0b1828032d8d243/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.210002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bfd8735327f4caf93453e2207bf7c36, user_id=8d2b0408dc084ea6a0b1828032d8d243, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:00.213171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9ba3eba02e493eb3058a1eb038fab9, user_id=2cd06e7233d74775abb4932266c8b268, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:00.216468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:00.222147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:00.224358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:00.364797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 868/1720] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:00 2022] PUT /identity/v3/projects/ee9ba3eba02e493eb3058a1eb038fab9/users/2cd06e7233d74775abb4932266c8b268/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:00.402231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 853/1721] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:00 2022] PUT /identity/v3/projects/7bfd8735327f4caf93453e2207bf7c36/users/8d2b0408dc084ea6a0b1828032d8d243/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:00.407339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:00.408236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:00.572377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5714de-66da-4a6b-a0a4-6f0f074d7628 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.586838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee1ee9-79a7-43a4-abb1-f462e6aeaafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.589722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1ee9-79a7-43a4-abb1-f462e6aeaafc admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.590051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1ee9-79a7-43a4-abb1-f462e6aeaafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.590834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee1ee9-79a7-43a4-abb1-f462e6aeaafc admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.591537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee1ee9-79a7-43a4-abb1-f462e6aeaafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:00.596895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acee1ee9-79a7-43a4-abb1-f462e6aeaafc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:00.628156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 869/1722] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:00.630680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.630988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.631253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:00.678826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69057de3-9507-416f-bbd5-d3fbb78a40bc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:00.720755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] MFA Rules not processed for user `8d2b0408dc084ea6a0b1828032d8d243`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:00.727841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:00.889427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29153fff-acb0-493f-9e51-e7d9301cdf69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:00.891856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153fff-acb0-493f-9e51-e7d9301cdf69 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:00.893602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153fff-acb0-493f-9e51-e7d9301cdf69 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:00.894111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29153fff-acb0-493f-9e51-e7d9301cdf69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.052331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153fff-acb0-493f-9e51-e7d9301cdf69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:01.056788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29153fff-acb0-493f-9e51-e7d9301cdf69 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:01.100646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 854/1723] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 703 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:01.102381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.108261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.108261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.175699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] MFA Rules not processed for user `76ea3ed3bb2b412093004c7e0f49c7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:01.177100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:01.227954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 870/1724] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 588 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:01.227954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:01.266027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31de6a4f-c7ac-4a47-883a-5b971fe267aa None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:01.286242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5559c6ed-805b-4457-b43c-ca48e7bb43e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:01.288213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559c6ed-805b-4457-b43c-ca48e7bb43e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.288498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559c6ed-805b-4457-b43c-ca48e7bb43e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.288779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5559c6ed-805b-4457-b43c-ca48e7bb43e8 admin admin] PATH_INFO: `/v3/projects/ee9ba3eba02e493eb3058a1eb038fab9/users/2cd06e7233d74775abb4932266c8b268/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.299945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559c6ed-805b-4457-b43c-ca48e7bb43e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee9ba3eba02e493eb3058a1eb038fab9, user_id=2cd06e7233d74775abb4932266c8b268, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:01.305991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:01.313123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5559c6ed-805b-4457-b43c-ca48e7bb43e8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:01.361637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 855/1725] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:01 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:01.361637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:01.361884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 871/1726] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:01 2022] PUT /identity/v3/projects/ee9ba3eba02e493eb3058a1eb038fab9/users/2cd06e7233d74775abb4932266c8b268/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:01.371964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.372258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.372539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.410505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a5522-4025-42bf-93d9-98f276bbcd41 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:01.429998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:01.490096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] MFA Rules not processed for user `2cd06e7233d74775abb4932266c8b268`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:01.492427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:01.620065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:01.620800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.624028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.624500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.626390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:01.636032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:01.761014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 872/1727] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:01.772766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:01.774589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:01.840338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7638565-8191-4c3f-b42b-6739ba1695a7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:01.845975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:01.848392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:01.848392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:01.848392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:01.855440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 856/1728] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:01 2022] POST /identity/v3/projects => generated 387 bytes in 494 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:01.857124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:01.930694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:01.934346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:01.970309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 873/1729] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:01.986212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a78dc-a3be-46a0-8235-9c4138208d34 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:02.004417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:02.005682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.006210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.006691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.007618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:02.009800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:02.157519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 857/1730] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:01 2022] POST /identity/v3/users => generated 300 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:02.158838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:02.179667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:02.187924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc84a-3b2b-4e32-8aca-23463692a500 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:02.203723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:02.209378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.212462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.217381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.217381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:02.223062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:02.277765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 874/1731] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:02.281633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:02.298552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd945de-2461-44de-b29f-97d53c588a72 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:02.314070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:02.315074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.315529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.315970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] PATH_INFO: `/v3/projects/b7890239d073449a9648ac4821174423/users/0c9e28a46b47439ca35323be4ac1a67d/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.347269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7890239d073449a9648ac4821174423, user_id=0c9e28a46b47439ca35323be4ac1a67d, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:02.350676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:02.477901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 858/1732] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:02 2022] PUT /identity/v3/projects/b7890239d073449a9648ac4821174423/users/0c9e28a46b47439ca35323be4ac1a67d/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:02.484009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:02.518520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69983808-e85b-4cd9-b012-101992c36f79 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:02.680597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345b778-9cc5-4811-b76e-68e008feb919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:02.681598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b778-9cc5-4811-b76e-68e008feb919 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.682047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b778-9cc5-4811-b76e-68e008feb919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.682683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345b778-9cc5-4811-b76e-68e008feb919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.683442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b778-9cc5-4811-b76e-68e008feb919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:02.685689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345b778-9cc5-4811-b76e-68e008feb919 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:02.698884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 875/1733] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:02.707746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:02.757154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c7b4b8-fc9a-4a9b-93e4-19ef03ee2f77 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:02.777086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b52e0-1996-42ce-a8a2-142f02827f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:02.777959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b52e0-1996-42ce-a8a2-142f02827f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.778343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b52e0-1996-42ce-a8a2-142f02827f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.778726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b52e0-1996-42ce-a8a2-142f02827f07 admin admin] PATH_INFO: `/v3/projects/b7890239d073449a9648ac4821174423/users/0c9e28a46b47439ca35323be4ac1a67d/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.807392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b52e0-1996-42ce-a8a2-142f02827f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7890239d073449a9648ac4821174423, user_id=0c9e28a46b47439ca35323be4ac1a67d, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:02.844538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b52e0-1996-42ce-a8a2-142f02827f07 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:02.875918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce22fd-2fe5-4053-839e-abb412d81eca None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.878094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce22fd-2fe5-4053-839e-abb412d81eca None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.879989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce22fd-2fe5-4053-839e-abb412d81eca None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.883068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 876/1734] 10.222.0.52 () {54 vars in 912 bytes} [Tue Aug 30 01:11:02 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:02.891087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.891616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.892099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.924877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 859/1735] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:02 2022] PUT /identity/v3/projects/b7890239d073449a9648ac4821174423/users/0c9e28a46b47439ca35323be4ac1a67d/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:02.938935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24e62-0020-4674-8775-6b6dcb4c7e9d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:02.939393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24e62-0020-4674-8775-6b6dcb4c7e9d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:02.939673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc24e62-0020-4674-8775-6b6dcb4c7e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:02.953579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:02.954859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:03.026637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ffc24e62-0020-4674-8775-6b6dcb4c7e9d None None] MFA Rules not processed for user `0c9e28a46b47439ca35323be4ac1a67d`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:03.032179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ffc24e62-0020-4674-8775-6b6dcb4c7e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:03.109358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 877/1736] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:11:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:03.133771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:03.135762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:03.153121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbd91ab-14fd-435d-bcb8-2a385ec037db None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:03.171993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:03.172640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:03.172905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:03.173219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:03.191213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:03.192685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:03.214447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 878/1737] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:03.264285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 860/1738] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:02 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:03.290230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:03.451269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe46f3-e84c-4d0f-9d13-0d66775fd3fb service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:03.491083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:03.491719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:03.492010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:03.492766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:03.566938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:03.571338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:03.646096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 879/1739] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:03 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:03.804645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ec4d6-c547-4407-ad10-e8af1489c0ee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:03.809283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ec4d6-c547-4407-ad10-e8af1489c0ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:03.809756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ec4d6-c547-4407-ad10-e8af1489c0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:03.934980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dc5ec4d6-c547-4407-ad10-e8af1489c0ee None None] MFA Rules not processed for user `942d175e7e2742728e66fea84c851038`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:03.936854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc5ec4d6-c547-4407-ad10-e8af1489c0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:04.151244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 861/1740] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:04.416246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:04.515331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:06.246467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:06.257795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abab8c5-29f1-4c63-b8a7-78a8a4e4c190 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:06.261761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:06.262476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:06.262763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:06.263037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:06.390881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:06.392966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:06.469517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 880/1741] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:06.937691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:07.027520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:07.040491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d93ce-027e-4f03-9f42-6b1282580346 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:07.041116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d93ce-027e-4f03-9f42-6b1282580346 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:07.042515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048d93ce-027e-4f03-9f42-6b1282580346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:07.099080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-048d93ce-027e-4f03-9f42-6b1282580346 None None] MFA Rules not processed for user `2ffe5c7cb44f476e913d64003105002c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:07.100978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-048d93ce-027e-4f03-9f42-6b1282580346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:07.260116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 862/1742] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:07 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:09.728429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:09.864247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:10.541596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:10.555611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb355e-747f-4434-8922-d3947961f27f service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:10.568865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:10.571370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:10.572493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:10.572932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:10.734598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309b5e-160b-4c63-b1fc-438515c83160 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:10.735396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309b5e-160b-4c63-b1fc-438515c83160 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:10.735823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9309b5e-160b-4c63-b1fc-438515c83160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:10.848710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e9309b5e-160b-4c63-b1fc-438515c83160 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:10.852657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9309b5e-160b-4c63-b1fc-438515c83160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:10.871249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:10.876125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:10.918805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 881/1743] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:11.003314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 863/1744] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:11:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:11.542389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:11.542389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:11.764156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7509e7-1a64-4ba1-90f8-17dac800325a service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:11.775122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:11.775773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:11.776053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:11.776327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:11.791082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:11.797292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:11.804666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 882/1745] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:11:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:11:11.875100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5b379-05dc-40ed-82c2-af83d30c1421 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:11.875100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5b379-05dc-40ed-82c2-af83d30c1421 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:11.875100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5b379-05dc-40ed-82c2-af83d30c1421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:11.933003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-97c5b379-05dc-40ed-82c2-af83d30c1421 None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:11.936256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-97c5b379-05dc-40ed-82c2-af83d30c1421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:11.949177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 864/1746] 10.222.0.74 () {58 vars in 1015 bytes} [Tue Aug 30 01:11:11 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:11.964787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:12.001349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb3a78-85f7-4651-8dfc-ecc3875b5fcb service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:12.029107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a96b4-2371-4466-8347-72a6f8636e87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:12.037503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a96b4-2371-4466-8347-72a6f8636e87 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:12.038638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a96b4-2371-4466-8347-72a6f8636e87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:12.040322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a96b4-2371-4466-8347-72a6f8636e87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:12.098724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a96b4-2371-4466-8347-72a6f8636e87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:12.105611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a96b4-2371-4466-8347-72a6f8636e87 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:12.160976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 883/1747] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:13.016274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:13.569653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:14.499547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503f90c-b073-4d15-a927-adf2116eba2d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:14.505123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503f90c-b073-4d15-a927-adf2116eba2d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:14.505123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9503f90c-b073-4d15-a927-adf2116eba2d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:14.505123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 865/1748] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:14.515475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:14.516142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:14.516920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:14.561769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:14.563196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:14.685377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 884/1749] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:11:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:14.705592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503f90c-b073-4d15-a927-adf2116eba2d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:14.733804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9503f90c-b073-4d15-a927-adf2116eba2d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:14.747506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:14.748682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:14.749480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:14.750025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:14.766348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:14.768345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:14.807304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 866/1750] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:15.516890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:16.018843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:16.534990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:17.145634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:19.471821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:19.487904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca13fc-252b-410e-89df-25999ec8b462 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:19.492237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808fee8e-ed79-416d-a175-00dccec818b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:19.493156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fee8e-ed79-416d-a175-00dccec818b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:19.493535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fee8e-ed79-416d-a175-00dccec818b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:19.493938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808fee8e-ed79-416d-a175-00dccec818b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:19.586432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fee8e-ed79-416d-a175-00dccec818b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:19.596027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808fee8e-ed79-416d-a175-00dccec818b8 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:19.600335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 885/1751] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:28.581356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:29.001685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:29.098130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b96cd3-f3a3-4ebd-a1f1-e5dcc7d8eda1 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:29.103369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:29.113937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:29.114493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:29.114638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4f099-a8a7-41a9-869d-83f28bf81ef2 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:29.115097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] PATH_INFO: `/v3/users/5d09f2a924f340bab640797a7bd61cdb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:29.115180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4f099-a8a7-41a9-869d-83f28bf81ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:29.115180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4f099-a8a7-41a9-869d-83f28bf81ef2 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:29.116533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 886/1752] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:11:29.118349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d09f2a924f340bab640797a7bd61cdb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:29.121347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:29.130066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:29.130606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:29.131124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:29.174993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:29.176364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:29.213506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 887/1753] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:11:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:29.475333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d09f2a924f340bab640797a7bd61cdb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:29.485322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d09f2a924f340bab640797a7bd61cdb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:29.591343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:29.594124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:29.614164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4384fa-4afa-4006-8b52-f2b8ef5a7eef None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:29.624260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c139d8f-a298-46ef-9b1b-2e2550737b8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:29.624907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c139d8f-a298-46ef-9b1b-2e2550737b8f service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:29.625195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c139d8f-a298-46ef-9b1b-2e2550737b8f service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:29.625471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c139d8f-a298-46ef-9b1b-2e2550737b8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:29.630271 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d09f2a924f340bab640797a7bd61cdb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:29.631643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d09f2a924f340bab640797a7bd61cdb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:29.637713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c139d8f-a298-46ef-9b1b-2e2550737b8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:29.641051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c139d8f-a298-46ef-9b1b-2e2550737b8f service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:29.653449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 888/1754] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:29.673470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d09f2a924f340bab640797a7bd61cdb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:29.682335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 867/1755] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:29 2022] DELETE /identity/v3/users/5d09f2a924f340bab640797a7bd61cdb => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:29.688219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:29.689959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:29.690426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:29.717494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:29.718969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:29.723883 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:29.819225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 889/1756] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:29.829025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:29.830551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:29.896705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99653d2c-4a01-4d31-83d9-c73f773c17ba admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:29.902098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:29.902098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:29.902098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:29.902098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:29.927949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:29.930729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:29.953544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 868/1757] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:30.577770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:30.579296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:30.591334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d6cc87-7e4f-4898-8e37-440f1dfafb3f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:30.599372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:30.599372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:30.599372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:30.599372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] PATH_INFO: `/v3/users/ca8478dec48d46db9ff19e715b60078e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:30.599934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca8478dec48d46db9ff19e715b60078e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:30.600804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:30.696815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:30.698393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:30.712884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4f23-8709-4367-afcb-11026c534af4 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:30.719138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:30.721168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:30.721823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:30.722499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] PATH_INFO: `/v3/projects/1064d44812324318a4805b8fe3c7f9c8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:30.724107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1064d44812324318a4805b8fe3c7f9c8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:30.726244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:30.754118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:30.913120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca8478dec48d46db9ff19e715b60078e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.917058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1064d44812324318a4805b8fe3c7f9c8', 'request_id': 'req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.920919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca8478dec48d46db9ff19e715b60078e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.921612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca8478dec48d46db9ff19e715b60078e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.922288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca8478dec48d46db9ff19e715b60078e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.924284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1064d44812324318a4805b8fe3c7f9c8', 'request_id': 'req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.930736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1064d44812324318a4805b8fe3c7f9c8', 'request_id': 'req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:30.935859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-17ad615e-9030-403e-a85b-22e5d615aa85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca8478dec48d46db9ff19e715b60078e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.018432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] The token cache is being invalidate because project 1064d44812324318a4805b8fe3c7f9c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:31.018743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.019878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.021577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 869/1758] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:30 2022] DELETE /identity/v3/projects/1064d44812324318a4805b8fe3c7f9c8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:31.028452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:31.030179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:31.070230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 890/1759] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:30 2022] DELETE /identity/v3/users/ca8478dec48d46db9ff19e715b60078e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:31.075207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:31.075518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:31.075948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:31.095209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:31.096414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:31.097882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d5570-a9ff-4268-aaec-3e4d6e3ca66f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:31.156838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 891/1760] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:31.157127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:31.157802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:31.158092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:31.158375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] PATH_INFO: `/v3/users/8d2b0408dc084ea6a0b1828032d8d243` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:31.159520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d2b0408dc084ea6a0b1828032d8d243)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:31.161611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:31.166799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:31.168172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:31.195928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:31.227479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e3b69a-ae39-41a3-a5b1-2d10a947efc3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:31.231096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:31.231706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:31.232529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:31.233358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:31.242776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:31.244046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:31.254691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 892/1761] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:31.412353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d2b0408dc084ea6a0b1828032d8d243'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.420058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d2b0408dc084ea6a0b1828032d8d243'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.546604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d2b0408dc084ea6a0b1828032d8d243'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.547404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d2b0408dc084ea6a0b1828032d8d243'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.568418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d2b0408dc084ea6a0b1828032d8d243'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:31.579323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 870/1762] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:31 2022] DELETE /identity/v3/users/8d2b0408dc084ea6a0b1828032d8d243 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:31.912028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:31.914213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:31.931144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a005bc4-98cb-4adc-9f35-0ef49d3bc085 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:32.013644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:32.014748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:32.015241 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:32.015534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] PATH_INFO: `/v3/projects/cd483df7bb9242ae8364aaa85c24b9f3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:32.016704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd483df7bb9242ae8364aaa85c24b9f3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:32.018832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:32.158319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:32.159873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:32.176716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd483df7bb9242ae8364aaa85c24b9f3', 'request_id': 'req-85f9e585-1561-493c-bdc5-1a602a5308a3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.189057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58405f-97ee-42fe-9397-e4e9009132b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:32.196050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:32.248981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:32.249540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:32.249859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:32.250140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] PATH_INFO: `/v3/projects/7bfd8735327f4caf93453e2207bf7c36` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:32.251262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bfd8735327f4caf93453e2207bf7c36)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:32.253189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:32.263941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd483df7bb9242ae8364aaa85c24b9f3', 'request_id': 'req-85f9e585-1561-493c-bdc5-1a602a5308a3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.270551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd483df7bb9242ae8364aaa85c24b9f3', 'request_id': 'req-85f9e585-1561-493c-bdc5-1a602a5308a3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.281535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] The token cache is being invalidate because project cd483df7bb9242ae8364aaa85c24b9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:32.281877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.282528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.284050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 893/1763] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:31 2022] DELETE /identity/v3/projects/cd483df7bb9242ae8364aaa85c24b9f3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:32.287900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:32.289235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:32.302592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f9e585-1561-493c-bdc5-1a602a5308a3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:32.307483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:32.308112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:32.308396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:32.308677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] PATH_INFO: `/v3/users/276d4fbe87d9459987048ffde9a5f36b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:32.310391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=276d4fbe87d9459987048ffde9a5f36b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:32.312456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:32.393070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bfd8735327f4caf93453e2207bf7c36', 'request_id': 'req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.399241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bfd8735327f4caf93453e2207bf7c36', 'request_id': 'req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.404864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bfd8735327f4caf93453e2207bf7c36', 'request_id': 'req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.538542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] The token cache is being invalidate because project 7bfd8735327f4caf93453e2207bf7c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:32.538887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.539757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.541410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 871/1764] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:32 2022] DELETE /identity/v3/projects/7bfd8735327f4caf93453e2207bf7c36 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:32.546224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:32.547695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:32.552456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '276d4fbe87d9459987048ffde9a5f36b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.561528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '276d4fbe87d9459987048ffde9a5f36b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.562500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '276d4fbe87d9459987048ffde9a5f36b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.563261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '276d4fbe87d9459987048ffde9a5f36b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.576407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '276d4fbe87d9459987048ffde9a5f36b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.617808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd34c2-4102-4edf-828f-fa5a1c7ce167 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:32.623870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:32.624629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:32.624926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:32.625219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] PATH_INFO: `/v3/users/0c9e28a46b47439ca35323be4ac1a67d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:32.626600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c9e28a46b47439ca35323be4ac1a67d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:32.631252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:32.661661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 894/1765] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:32 2022] DELETE /identity/v3/users/276d4fbe87d9459987048ffde9a5f36b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:32.838524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c9e28a46b47439ca35323be4ac1a67d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.849504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c9e28a46b47439ca35323be4ac1a67d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.935535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:32.979009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c9e28a46b47439ca35323be4ac1a67d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.979692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c9e28a46b47439ca35323be4ac1a67d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:32.998494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c9e28a46b47439ca35323be4ac1a67d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.010401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 872/1766] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:32 2022] DELETE /identity/v3/users/0c9e28a46b47439ca35323be4ac1a67d => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:33.476011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:33.477698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:33.570301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef254c-18ab-4e8b-9d72-84ec70544fe3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:33.644972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:33.645598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:33.645929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:33.646213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] PATH_INFO: `/v3/projects/f7e3694c8521423583aaaefdbe7d3bee` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:33.647507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e3694c8521423583aaaefdbe7d3bee)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:33.649928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:33.730994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:33.733231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:33.750750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0752966f-eeee-4447-84ab-fd5e0f92a527 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:33.755894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:33.756725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:33.757011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:33.757288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] PATH_INFO: `/v3/projects/b7890239d073449a9648ac4821174423` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:33.761708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7890239d073449a9648ac4821174423)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:33.764019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:33.812927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e3694c8521423583aaaefdbe7d3bee', 'request_id': 'req-1ef61d54-ad50-40a3-b821-d86003120194'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.910782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7890239d073449a9648ac4821174423', 'request_id': 'req-99b77110-9c25-4796-9083-7386668a53c1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.919774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7890239d073449a9648ac4821174423', 'request_id': 'req-99b77110-9c25-4796-9083-7386668a53c1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.933465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7890239d073449a9648ac4821174423', 'request_id': 'req-99b77110-9c25-4796-9083-7386668a53c1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.952671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e3694c8521423583aaaefdbe7d3bee', 'request_id': 'req-1ef61d54-ad50-40a3-b821-d86003120194'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.961330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e3694c8521423583aaaefdbe7d3bee', 'request_id': 'req-1ef61d54-ad50-40a3-b821-d86003120194'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.974629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] The token cache is being invalidate because project f7e3694c8521423583aaaefdbe7d3bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:33.975169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.976009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef61d54-ad50-40a3-b821-d86003120194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:33.978018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 895/1767] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:33 2022] DELETE /identity/v3/projects/f7e3694c8521423583aaaefdbe7d3bee => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:33.983015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5652-9b4e-43df-a43c-046ab5837b42 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:33.983655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5652-9b4e-43df-a43c-046ab5837b42 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:33.984065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaed5652-9b4e-43df-a43c-046ab5837b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.015382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-eaed5652-9b4e-43df-a43c-046ab5837b42 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:34.016945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-eaed5652-9b4e-43df-a43c-046ab5837b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:34.057812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] The token cache is being invalidate because project b7890239d073449a9648ac4821174423 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:34.058410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:34.059328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-99b77110-9c25-4796-9083-7386668a53c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:34.061428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 873/1768] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:33 2022] DELETE /identity/v3/projects/b7890239d073449a9648ac4821174423 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:34.070984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0c16d-3252-4fa1-945d-88669c174ee8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.071675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0c16d-3252-4fa1-945d-88669c174ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.072379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c0c16d-3252-4fa1-945d-88669c174ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.094220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 896/1769] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:34.111218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-79c0c16d-3252-4fa1-945d-88669c174ee8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:34.120772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.120772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.121798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.123948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-79c0c16d-3252-4fa1-945d-88669c174ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:34.134387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 874/1770] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:34.147673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.148176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.148792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.170929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:34.174788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:34.178881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:34.179667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:34.190284 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:34.201177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 875/1771] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:34.201949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 897/1772] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:34.208152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.208908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.227712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953cf4f-3dad-4298-b29a-ec77ae8de974 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.234065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53157e2f-df64-4f3c-ae06-471e74c4bf69 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.234255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.234839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.235196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.235549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.236593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:11:34.236910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.239904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.243270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.244664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.245099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.245633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.250958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:11:34.252615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.254889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.256136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 898/1773] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:11:34.262089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.272839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 876/1774] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:11:34.275139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb8c65-4e46-4d63-a7b7-33ae836a4ab0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.277095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.284345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.285610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.285938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.286217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.286966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.289571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.290333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5bd0a-2390-4cfd-9cb6-d01d0138e69a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.296104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.299081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.299081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.299081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.299081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.300895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.418122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 899/1775] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/projects => generated 387 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:34.419635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.463325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 877/1776] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/projects => generated 399 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:34.467244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.526505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea6d915-c1d6-4a0e-8fab-c1ae70a0fb1e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.609464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6cae2-5a80-4ed0-9cb7-a08d08bd8d0d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.616349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.617051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.617473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.617772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.618448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.621448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.633881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.635012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.635302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.635709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.636819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.641477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.777691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 878/1777] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/users => generated 300 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:34.783088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.804356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 900/1778] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:34 2022] POST /identity/v3/users => generated 294 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:34.808816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.819764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f540c73d-10af-4244-8b09-4882472c6896 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.836342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97139c64-0c59-49c5-bab8-fddc9c590c6d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.838634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.839696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.839696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.839880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.841140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.843129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.858695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.859328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.859611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.859891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.860543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.862208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.873409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 879/1779] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:34.876616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.885118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 901/1780] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:34.886533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.892238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a61d8-31fa-43ad-beca-2158831ca77b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.899865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e3128a-77af-45cb-9188-9c2303acebeb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.904377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.905684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.907896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.907896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.907896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.909866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.913177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.916270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.916791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.917258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.923801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:34.927185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:34.937807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 880/1781] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:34.941604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.952933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec4a5b2-9753-4c38-86f6-99a43f49ae90 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.957168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 902/1782] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:34.966456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:34.982758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8657de-df7c-4977-b9ba-aa680d4b1fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:34.982758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8657de-df7c-4977-b9ba-aa680d4b1fe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:34.982758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8657de-df7c-4977-b9ba-aa680d4b1fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:34.984091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8657de-df7c-4977-b9ba-aa680d4b1fe3 admin admin] PATH_INFO: `/v3/projects/b47f546baf7341da9abda1a754006d1a/users/46170215c302454598721e29fd81370e/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:34.987478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473911d-1e5f-4a33-8767-9186e617c037 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:34.996019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007345f-db29-45c5-bf3a-03f5a2335395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.034014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007345f-db29-45c5-bf3a-03f5a2335395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.034014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007345f-db29-45c5-bf3a-03f5a2335395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.034014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007345f-db29-45c5-bf3a-03f5a2335395 admin admin] PATH_INFO: `/v3/projects/5f2ce2da90194bc88cd90179906ec9c4/users/320ce752a57a4b9b89b403030eb5d50a/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.034482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8657de-df7c-4977-b9ba-aa680d4b1fe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b47f546baf7341da9abda1a754006d1a, user_id=46170215c302454598721e29fd81370e, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.034482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8657de-df7c-4977-b9ba-aa680d4b1fe3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.034662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007345f-db29-45c5-bf3a-03f5a2335395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f2ce2da90194bc88cd90179906ec9c4, user_id=320ce752a57a4b9b89b403030eb5d50a, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.034662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007345f-db29-45c5-bf3a-03f5a2335395 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.087136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:35.197257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 881/1783] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:34 2022] PUT /identity/v3/projects/b47f546baf7341da9abda1a754006d1a/users/46170215c302454598721e29fd81370e/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:35.204030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 903/1784] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:34 2022] PUT /identity/v3/projects/5f2ce2da90194bc88cd90179906ec9c4/users/320ce752a57a4b9b89b403030eb5d50a/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:35.215104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.215320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.215551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.215661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.215838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.216187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.246683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] MFA Rules not processed for user `320ce752a57a4b9b89b403030eb5d50a`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:35.249214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:35.249624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] MFA Rules not processed for user `46170215c302454598721e29fd81370e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:35.250939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:35.321338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 882/1785] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:35 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:35.324964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 904/1786] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:35.326669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.329984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.404783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4cd0-8905-4b49-b449-00fb4bb51ee1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.408673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030271ec-1c1d-42bd-a27b-faded11c4e85 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.410780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.411419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.411702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.411974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.412759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.414168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.415269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.415801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.416245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.416678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.417524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.421026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.553198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 883/1787] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:35 2022] POST /identity/v3/projects => generated 399 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:35.554036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.620556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 905/1788] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:35 2022] POST /identity/v3/projects => generated 387 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:35.628445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.631623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec7e497-ba8d-42b2-882a-a37e2439b9bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.722276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.723280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.725021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.726707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cdb3b0-21bf-42d2-9db1-8418a281926f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.727563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.729116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.733236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.739499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.740142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.740429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.740710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.741656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.744640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.865006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 884/1790] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:35 2022] POST /identity/v3/users => generated 306 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:35.865668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 906/1790] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:35 2022] POST /identity/v3/users => generated 300 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:35.872984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.878285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.890102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1838c5-7ad4-47c7-a9b4-d19876c06dfc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.898694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.899328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.899611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.899726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0959dad-c4b7-4395-9b6d-6f10b84c7b87 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.899883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.900527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.906450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.914516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.915144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.915591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.915875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.916571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.920799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.928540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 885/1791] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:35.930144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.931217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 907/1792] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:35 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:35.935193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:35.945454 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1dc13-f576-4718-8875-2b3f2e08f2d9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.946568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544e9d9-687c-41ed-b584-1921fed8fb5f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:35.950443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.951110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.951428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.951669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] PATH_INFO: `/v3/projects/36cc862bdd514b668ea3e11a9ff02b2f/users/fb9c206b72f946c1a35df8153edff24f/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.952480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:35.953459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:35.953796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:35.954095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] PATH_INFO: `/v3/projects/6e63392c1a2e44a49daca9370ff56dc3/users/14fb8a3201014100a46629ee71a4ca1e/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:35.973956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cc862bdd514b668ea3e11a9ff02b2f, user_id=fb9c206b72f946c1a35df8153edff24f, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.977394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:35.989059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e63392c1a2e44a49daca9370ff56dc3, user_id=14fb8a3201014100a46629ee71a4ca1e, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:35.991776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.109118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 886/1793] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:35 2022] PUT /identity/v3/projects/36cc862bdd514b668ea3e11a9ff02b2f/users/fb9c206b72f946c1a35df8153edff24f/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:36.113488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 908/1794] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:35 2022] PUT /identity/v3/projects/6e63392c1a2e44a49daca9370ff56dc3/users/14fb8a3201014100a46629ee71a4ca1e/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:36.113671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.113787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.192076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eada8f5-cb2d-4dc8-8a47-4714839c9f2d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.243778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e370825f-1f2d-4696-a9c1-eaf71d83b3a6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.249620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.250417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.250795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.251642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.252394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.254368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.270574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 909/1795] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:36.272125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.286436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.287125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.287414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.287690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.288446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.290498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.298829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f742d4-1268-4461-9e02-7e97230497fd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.303308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 887/1796] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:36.307714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.311968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a102a4-1008-4b3a-bcb5-7751b38615b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.312943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a102a4-1008-4b3a-bcb5-7751b38615b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.313381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a102a4-1008-4b3a-bcb5-7751b38615b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.313863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a102a4-1008-4b3a-bcb5-7751b38615b3 admin admin] PATH_INFO: `/v3/projects/6e63392c1a2e44a49daca9370ff56dc3/users/14fb8a3201014100a46629ee71a4ca1e/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.316404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a102a4-1008-4b3a-bcb5-7751b38615b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e63392c1a2e44a49daca9370ff56dc3, user_id=14fb8a3201014100a46629ee71a4ca1e, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.317814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:36.320032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a102a4-1008-4b3a-bcb5-7751b38615b3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.323797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cf0197-fc60-40a5-aaa8-ec0ef10c68ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.331065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006ff4eb-77f7-41ac-a0ee-3dd79a44997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.331950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ff4eb-77f7-41ac-a0ee-3dd79a44997d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.332339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ff4eb-77f7-41ac-a0ee-3dd79a44997d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.332722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006ff4eb-77f7-41ac-a0ee-3dd79a44997d admin admin] PATH_INFO: `/v3/projects/36cc862bdd514b668ea3e11a9ff02b2f/users/fb9c206b72f946c1a35df8153edff24f/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.335011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ff4eb-77f7-41ac-a0ee-3dd79a44997d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36cc862bdd514b668ea3e11a9ff02b2f, user_id=fb9c206b72f946c1a35df8153edff24f, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.338169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006ff4eb-77f7-41ac-a0ee-3dd79a44997d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.338420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 910/1797] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:36 2022] PUT /identity/v3/projects/6e63392c1a2e44a49daca9370ff56dc3/users/14fb8a3201014100a46629ee71a4ca1e/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:36.362713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.362713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.362713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.364840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 888/1798] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:36 2022] PUT /identity/v3/projects/36cc862bdd514b668ea3e11a9ff02b2f/users/fb9c206b72f946c1a35df8153edff24f/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:36.377891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.378413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.379136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.382957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] MFA Rules not processed for user `14fb8a3201014100a46629ee71a4ca1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:36.408044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:36.479929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] MFA Rules not processed for user `fb9c206b72f946c1a35df8153edff24f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:36.482567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:36.525783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 911/1799] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:36.538515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.540372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:36.585889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 889/1800] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:36 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:36.593541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.595003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:36.625390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4b02-b06a-490b-ab78-208373aae4cf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.629023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.629635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.629945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.630217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.640265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.641707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.651212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 912/1801] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:36.655172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e463d03e-2bf4-4c53-baa1-e39f6275d510 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.659117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.659792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.660254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.660347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.670656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.672255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.684268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 890/1802] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:36 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:36.720130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.784089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20703119-65a1-4792-a279-581c1f872a1c service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.788687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.789333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.789594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.789902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] PATH_INFO: `/v3/users/320ce752a57a4b9b89b403030eb5d50a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:36.791028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] RBAC: Authorizing `identity:delete_user(user_id=320ce752a57a4b9b89b403030eb5d50a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:36.792920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:36.959676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:36.970224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93625b0a-af3c-4198-9c33-d71da4f6a9b5 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:36.974306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8d973-899c-40e1-9978-22a9c246009d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:36.975187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8d973-899c-40e1-9978-22a9c246009d service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:36.975537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8d973-899c-40e1-9978-22a9c246009d service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:36.975903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8d973-899c-40e1-9978-22a9c246009d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:37.051833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8d973-899c-40e1-9978-22a9c246009d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:37.055186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '320ce752a57a4b9b89b403030eb5d50a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:37.057555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8d973-899c-40e1-9978-22a9c246009d service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:37.072896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '320ce752a57a4b9b89b403030eb5d50a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:37.073586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '320ce752a57a4b9b89b403030eb5d50a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:37.074240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '320ce752a57a4b9b89b403030eb5d50a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:37.077984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 891/1803] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:36 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:37.093231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '320ce752a57a4b9b89b403030eb5d50a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:37.182649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 913/1804] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:36 2022] DELETE /identity/v3/users/320ce752a57a4b9b89b403030eb5d50a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:37.188325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d974a-2fc1-4753-9670-c891e149135b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:37.188906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d974a-2fc1-4753-9670-c891e149135b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:37.189343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d974a-2fc1-4753-9670-c891e149135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:37.214617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-307d974a-2fc1-4753-9670-c891e149135b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:37.216760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-307d974a-2fc1-4753-9670-c891e149135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:37.284760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 892/1805] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:37.294650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:37.296129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:37.356333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cf894-ab1e-427c-ac13-44fbd4d20823 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:37.360374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:37.361065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:37.361416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:37.361701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:37.369296 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:37.373235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:37.381579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:37.384690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 914/1806] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:37.886289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d974a-2fc1-4753-9670-c891e149135b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:37.887415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-307d974a-2fc1-4753-9670-c891e149135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:37.900366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d974a-2fc1-4753-9670-c891e149135b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:37.908091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:37.908835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:37.909260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:37.909544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] PATH_INFO: `/v3/projects/5f2ce2da90194bc88cd90179906ec9c4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:37.910817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f2ce2da90194bc88cd90179906ec9c4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:37.913320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:38.099110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f2ce2da90194bc88cd90179906ec9c4', 'request_id': 'req-37357eb4-74da-4308-8de4-745540d90cfb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.105491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f2ce2da90194bc88cd90179906ec9c4', 'request_id': 'req-37357eb4-74da-4308-8de4-745540d90cfb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.112103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f2ce2da90194bc88cd90179906ec9c4', 'request_id': 'req-37357eb4-74da-4308-8de4-745540d90cfb'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.188976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] The token cache is being invalidate because project 5f2ce2da90194bc88cd90179906ec9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:38.189291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.189936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.191436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 893/1807] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:37 2022] DELETE /identity/v3/projects/5f2ce2da90194bc88cd90179906ec9c4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:38.195837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:38.205828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fcf1a-bd35-4dd0-af03-9f9f73a0f1b5 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:38.264096 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:38.264748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:38.265027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:38.265301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] PATH_INFO: `/v3/users/14fb8a3201014100a46629ee71a4ca1e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:38.266665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=14fb8a3201014100a46629ee71a4ca1e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:38.268631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:38.543526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14fb8a3201014100a46629ee71a4ca1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.553513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14fb8a3201014100a46629ee71a4ca1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.553989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14fb8a3201014100a46629ee71a4ca1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.555740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14fb8a3201014100a46629ee71a4ca1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.570403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a5802f36-f2e3-430b-bc90-1737ca17cd4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14fb8a3201014100a46629ee71a4ca1e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:38.696468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 915/1808] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:38 2022] DELETE /identity/v3/users/14fb8a3201014100a46629ee71a4ca1e => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:38.781298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:38.969679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:39.257136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.258834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:39.273678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37357eb4-74da-4308-8de4-745540d90cfb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:39.337081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:39.337747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.338122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.338458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] PATH_INFO: `/v3/projects/6e63392c1a2e44a49daca9370ff56dc3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.339591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e63392c1a2e44a49daca9370ff56dc3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:39.342191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:39.461859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e63392c1a2e44a49daca9370ff56dc3', 'request_id': 'req-bae63b86-4c21-4399-a58f-f41456a27373'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.469077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e63392c1a2e44a49daca9370ff56dc3', 'request_id': 'req-bae63b86-4c21-4399-a58f-f41456a27373'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.475002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e63392c1a2e44a49daca9370ff56dc3', 'request_id': 'req-bae63b86-4c21-4399-a58f-f41456a27373'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.626964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] The token cache is being invalidate because project 6e63392c1a2e44a49daca9370ff56dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:39.627265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.628067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bae63b86-4c21-4399-a58f-f41456a27373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:39.629830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 894/1809] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:39 2022] DELETE /identity/v3/projects/6e63392c1a2e44a49daca9370ff56dc3 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:39.641344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.641344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.641344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.663143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:39.664652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:39.728286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 916/1810] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:39.737010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.737387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.737748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.758837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:39.761596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:39.769194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 895/1811] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:39.773490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.784681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc9ee27-f70e-4ebd-acce-b244d5cd74bf None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:39.788062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:39.791286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:39.792010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.792172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.792448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.793106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:11:39.793411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:39.795189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:39.808369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 917/1812] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:11:39.816374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.837532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb15924-e397-4089-bb29-e49f8d516871 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:39.843135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:39.843764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:39.844044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:39.844318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:39.844971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:39.847856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:39.854929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:39.977134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 896/1813] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:39 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:39.985336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:39.995528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50fd1e-5dd6-433d-b8c0-791a449bd5f7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.111193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf001b-4bcc-48c4-b498-5f7b0b3ef64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.111813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf001b-4bcc-48c4-b498-5f7b0b3ef64a admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.112093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf001b-4bcc-48c4-b498-5f7b0b3ef64a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.112389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf001b-4bcc-48c4-b498-5f7b0b3ef64a admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.113006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf001b-4bcc-48c4-b498-5f7b0b3ef64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.114960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf001b-4bcc-48c4-b498-5f7b0b3ef64a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:40.243496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 918/1814] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:39 2022] POST /identity/v3/users => generated 290 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:40.247686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:40.258959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741aaf-ed83-4b05-87b4-c613e2562cbd None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.259249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741aaf-ed83-4b05-87b4-c613e2562cbd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.259645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1741aaf-ed83-4b05-87b4-c613e2562cbd None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.260514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e386d3-4a22-4313-990e-f9c57f5d9546 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.262874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 919/1815] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:11:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:11:40.268332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.268980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.269353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.274026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.274026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.274289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.274289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.274704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.278147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:40.291002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 897/1816] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:40.295237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:40.299287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:40.300522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:40.306517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed68b3-893d-45ff-b9fa-6b9fce707a21 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.311926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.312553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.312858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.313321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.314207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.316167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:40.328533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 898/1817] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:40.332809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:40.345339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11666f6f-c654-42d9-bed0-8332d42a395b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.350784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.351458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.351812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.352089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] PATH_INFO: `/v3/projects/4c413b018e5544548d62af565e6c4b8d/users/17517a3f1fda4e1db469a30931cad961/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.368001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c413b018e5544548d62af565e6c4b8d, user_id=17517a3f1fda4e1db469a30931cad961, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.370684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:40.384666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 920/1818] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:11:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:40.410686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:40.412142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:40.427404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8446e519-bd12-4403-b51c-a4ef1b364b8b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.434271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a64bb-210c-4fd9-9bc6-8430969cf88e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.434964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a64bb-210c-4fd9-9bc6-8430969cf88e service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.435249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a64bb-210c-4fd9-9bc6-8430969cf88e service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.435525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a64bb-210c-4fd9-9bc6-8430969cf88e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.444992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a64bb-210c-4fd9-9bc6-8430969cf88e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.446627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108a64bb-210c-4fd9-9bc6-8430969cf88e service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:40.457277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 921/1819] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:40.464305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 899/1820] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:40 2022] PUT /identity/v3/projects/4c413b018e5544548d62af565e6c4b8d/users/17517a3f1fda4e1db469a30931cad961/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:40.478691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.478987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.479320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.512888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] MFA Rules not processed for user `17517a3f1fda4e1db469a30931cad961`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:40.514223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:40.584717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 922/1821] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:40.588820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:40.652970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a97f0-27ce-4cb4-881e-67aa0296c599 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.658256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.659010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.659292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.659566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.660197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.662122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:40.785033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 900/1822] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:40 2022] POST /identity/v3/projects => generated 379 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:40.792647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:40.812721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7b94a-cac3-41e0-8ad6-3d4492478f46 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:40.881797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:40.882423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:40.882705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:40.882980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:40.883815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:40.889881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.016062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 923/1823] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:40 2022] POST /identity/v3/users => generated 296 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:41.019950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:41.041634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b3cbb-55b2-475e-ae32-6e340177754b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:41.063730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:41.063730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.063730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.064983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.069967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:41.071693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.098923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 901/1824] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:41.100232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:41.119757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe11412-acb4-414f-90b3-4cb336edd6a7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:41.147740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:41.147740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.147740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.147740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] PATH_INFO: `/v3/projects/2723e0408e4a48ba85883c34d5499fd0/users/99fdd1669a464be5b3a88830c63b0af9/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.161572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] RBAC: Authorizing `identity:create_grant(project_id=2723e0408e4a48ba85883c34d5499fd0, user_id=99fdd1669a464be5b3a88830c63b0af9, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:41.161572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.312458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:41.321241 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 924/1825] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:41 2022] PUT /identity/v3/projects/2723e0408e4a48ba85883c34d5499fd0/users/99fdd1669a464be5b3a88830c63b0af9/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:41.322790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:41.342564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370a3db-5576-45a2-9966-7754490034fb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:41.426237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb3536-e850-4235-8e8a-8a3332844f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:41.427101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3536-e850-4235-8e8a-8a3332844f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.427487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3536-e850-4235-8e8a-8a3332844f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.427904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb3536-e850-4235-8e8a-8a3332844f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.428685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb3536-e850-4235-8e8a-8a3332844f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:41.430675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb3536-e850-4235-8e8a-8a3332844f18 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.462695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 902/1826] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:41.463976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:41.476349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fab72-4228-49e5-b7af-ffe680e253da admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:41.481274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:41.486249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.486550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.486836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] PATH_INFO: `/v3/projects/2723e0408e4a48ba85883c34d5499fd0/users/99fdd1669a464be5b3a88830c63b0af9/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.491214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2723e0408e4a48ba85883c34d5499fd0, user_id=99fdd1669a464be5b3a88830c63b0af9, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:41.494023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.507001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 925/1827] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:41 2022] PUT /identity/v3/projects/2723e0408e4a48ba85883c34d5499fd0/users/99fdd1669a464be5b3a88830c63b0af9/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:41.520607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292478-2497-4801-a825-6304d9482509 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.521429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292478-2497-4801-a825-6304d9482509 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.522562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10292478-2497-4801-a825-6304d9482509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.569488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:41.588185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-10292478-2497-4801-a825-6304d9482509 None None] MFA Rules not processed for user `99fdd1669a464be5b3a88830c63b0af9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:41.590061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-10292478-2497-4801-a825-6304d9482509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:41.680982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 903/1828] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:41.693088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:41.695305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:41.779964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde8061b-a686-41b2-bbc6-56a8de988b6d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:41.783816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dffd2c-469a-4a12-be61-402013b61959 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:41.784442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffd2c-469a-4a12-be61-402013b61959 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:41.784715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffd2c-469a-4a12-be61-402013b61959 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:41.785144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dffd2c-469a-4a12-be61-402013b61959 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:41.795707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dffd2c-469a-4a12-be61-402013b61959 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:41.797275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dffd2c-469a-4a12-be61-402013b61959 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:41.807625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 926/1829] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:42.832983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292478-2497-4801-a825-6304d9482509 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:42.912473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10292478-2497-4801-a825-6304d9482509 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:42.918257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:42.918833 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:42.919701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:42.919985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] PATH_INFO: `/v3/users/17517a3f1fda4e1db469a30931cad961` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:42.921461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=17517a3f1fda4e1db469a30931cad961)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:42.923733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:43.191130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17517a3f1fda4e1db469a30931cad961'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:43.198858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17517a3f1fda4e1db469a30931cad961'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:43.290344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17517a3f1fda4e1db469a30931cad961'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:43.291000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17517a3f1fda4e1db469a30931cad961'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:43.310278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17517a3f1fda4e1db469a30931cad961'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:43.319178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 904/1830] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:42 2022] DELETE /identity/v3/users/17517a3f1fda4e1db469a30931cad961 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:43.324124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:43.324613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:43.324691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:43.372347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:43.382446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:43.479599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 927/1831] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:43.490871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:43.492579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:43.588210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1af33a9-b4da-47f1-95cd-a8630a5ecdeb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:43.592639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:43.593275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:43.593558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:43.593877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:43.606267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:43.608817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:43.638418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 905/1832] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:44.207632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:44.214914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:44.217020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:44.229574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6373-108a-4f2d-8628-45624df3109c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:44.240093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:44.240758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:44.241184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:44.241314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] PATH_INFO: `/v3/projects/4c413b018e5544548d62af565e6c4b8d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:44.242670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c413b018e5544548d62af565e6c4b8d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:44.244604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:44.389719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c413b018e5544548d62af565e6c4b8d', 'request_id': 'req-2552a40f-b5fe-4415-9bc4-ff865f36e994'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:44.484680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c413b018e5544548d62af565e6c4b8d', 'request_id': 'req-2552a40f-b5fe-4415-9bc4-ff865f36e994'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:44.492282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c413b018e5544548d62af565e6c4b8d', 'request_id': 'req-2552a40f-b5fe-4415-9bc4-ff865f36e994'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:44.503873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] The token cache is being invalidate because project 4c413b018e5544548d62af565e6c4b8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:44.504170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:44.504844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:44.506436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 928/1833] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:44 2022] DELETE /identity/v3/projects/4c413b018e5544548d62af565e6c4b8d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:44.510607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:44.521026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3be722-3965-4700-bb57-f5007a927038 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:44.582617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:44.583259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:44.583570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:44.583850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] PATH_INFO: `/v3/users/99fdd1669a464be5b3a88830c63b0af9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:44.584986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99fdd1669a464be5b3a88830c63b0af9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:44.586981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:45.093129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99fdd1669a464be5b3a88830c63b0af9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.101981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99fdd1669a464be5b3a88830c63b0af9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.208736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99fdd1669a464be5b3a88830c63b0af9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.209339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99fdd1669a464be5b3a88830c63b0af9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.225966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99fdd1669a464be5b3a88830c63b0af9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.229576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 906/1834] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:44 2022] DELETE /identity/v3/users/99fdd1669a464be5b3a88830c63b0af9 => generated 0 bytes in 719 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:45.378195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:45.379681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:45.393623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552a40f-b5fe-4415-9bc4-ff865f36e994 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:45.454966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:45.455641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:45.455980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:45.456254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] PATH_INFO: `/v3/users/46170215c302454598721e29fd81370e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:45.457611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=46170215c302454598721e29fd81370e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:45.459733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:45.767362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:45.768988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:45.778001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46170215c302454598721e29fd81370e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.800277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e8eb7-6c35-4798-bd1a-bb6799634e80 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:45.806139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46170215c302454598721e29fd81370e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.806959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46170215c302454598721e29fd81370e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.807737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46170215c302454598721e29fd81370e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.821836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:45.822587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:45.822881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:45.823168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] PATH_INFO: `/v3/projects/2723e0408e4a48ba85883c34d5499fd0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:45.824638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46170215c302454598721e29fd81370e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:45.825456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2723e0408e4a48ba85883c34d5499fd0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:45.829810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.032401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 929/1835] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:45 2022] DELETE /identity/v3/users/46170215c302454598721e29fd81370e => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:46.033943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.050001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.050392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2723e0408e4a48ba85883c34d5499fd0', 'request_id': 'req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.050392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2723e0408e4a48ba85883c34d5499fd0', 'request_id': 'req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.055281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2723e0408e4a48ba85883c34d5499fd0', 'request_id': 'req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.106631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:46.121426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cba92-596b-4508-a247-0f18e008a88d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.126177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.126854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.127165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.127418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] PATH_INFO: `/v3/users/88c21dfd715c4befbb040ec9b74a15e5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.128674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c21dfd715c4befbb040ec9b74a15e5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.130744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.168350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] The token cache is being invalidate because project 2723e0408e4a48ba85883c34d5499fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:46.168647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.169656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e5ac599b-fa45-4a25-bc9b-7a7c3623dd9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.182192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 907/1836] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:45 2022] DELETE /identity/v3/projects/2723e0408e4a48ba85883c34d5499fd0 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:46.182192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a56ddc-736f-4583-beac-55c9575514e1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.182192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a56ddc-736f-4583-beac-55c9575514e1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.182192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a56ddc-736f-4583-beac-55c9575514e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.193709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-43a56ddc-736f-4583-beac-55c9575514e1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:46.194975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43a56ddc-736f-4583-beac-55c9575514e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.204557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 908/1837] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:46.206293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.206759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.213584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.234483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:46.235709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.243801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 909/1838] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:46.244788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.250629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.316864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1512eb5-0add-4f0d-890a-cdcafa0dbd0c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.320711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81e9f7-3fb2-4b70-b5d9-396161e1cfcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.321403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81e9f7-3fb2-4b70-b5d9-396161e1cfcf service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.321782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81e9f7-3fb2-4b70-b5d9-396161e1cfcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.322066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81e9f7-3fb2-4b70-b5d9-396161e1cfcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.333035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81e9f7-3fb2-4b70-b5d9-396161e1cfcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.335346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81e9f7-3fb2-4b70-b5d9-396161e1cfcf service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.349096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 910/1839] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:46.350753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.351400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.351831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.377066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:46.378619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.392404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 911/1840] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:46.398146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.408909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797001a-6ef0-4e79-8c9b-ecdb37e9a6fc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.414893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c21dfd715c4befbb040ec9b74a15e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.417276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.417985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.418884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.419189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.420106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:11:46.420419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.422191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.424969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c21dfd715c4befbb040ec9b74a15e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.426383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c21dfd715c4befbb040ec9b74a15e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.427175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c21dfd715c4befbb040ec9b74a15e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.451730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 912/1841] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:11:46.461501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ce15f241-0d55-494a-9491-a822b6e23a03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c21dfd715c4befbb040ec9b74a15e5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:46.461800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.465097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.490070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa526-3b6c-40c6-8a31-c8cee2749b4f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.496129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.502678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.503163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.503771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.504585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.507381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.612895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 930/1842] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:46 2022] DELETE /identity/v3/users/88c21dfd715c4befbb040ec9b74a15e5 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:46.622540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.623028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.623448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.644874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:46.646175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.656207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 913/1843] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:46 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:46.660397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.724604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 931/1844] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:46.735239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.738196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:46.775461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc2c7b-ec81-4c3e-913d-18a52cf1f0ed admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.781796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.782661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.783006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.783339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.784077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.786908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.841568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bc098e-170b-4641-8a0d-3e549c848ad2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.910835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.911513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.911889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.912175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.923757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.925260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.936580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 932/1845] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:46.944111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 914/1846] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:46 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:46.948342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.958937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76328d-9504-4f22-a6a6-3c0023d27d39 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.963886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:46.964474 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:46.964666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:46.965011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:46.965718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:46.967713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:46.980088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 933/1847] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:46.985458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.993242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:46.998111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82935f55-978c-4719-b43f-2811efbe3a04 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:46.999790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:47.010880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:47.011973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.012322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.012723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b592e1-24a0-4c90-895d-d616ff92e842 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:47.012956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.013856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:47.017551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:47.019976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:47.020686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.020969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.021298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] PATH_INFO: `/v3/projects/b47f546baf7341da9abda1a754006d1a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.022717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b47f546baf7341da9abda1a754006d1a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:47.025432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:47.037074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 915/1848] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:47.041386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:47.056906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24aca8-f49d-46e9-ad47-6ee10a39cdb9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:47.068103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dec0d6-ecda-4104-9817-24c4fb6be6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:47.070577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dec0d6-ecda-4104-9817-24c4fb6be6c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.070907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dec0d6-ecda-4104-9817-24c4fb6be6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.072784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dec0d6-ecda-4104-9817-24c4fb6be6c0 admin admin] PATH_INFO: `/v3/projects/43f140e0f58f42ae96e16f80b480621c/users/1b225b55d18740c2833cade52edc556e/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.114027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dec0d6-ecda-4104-9817-24c4fb6be6c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43f140e0f58f42ae96e16f80b480621c, user_id=1b225b55d18740c2833cade52edc556e, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:47.117829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dec0d6-ecda-4104-9817-24c4fb6be6c0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:47.233078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 916/1849] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:47 2022] PUT /identity/v3/projects/43f140e0f58f42ae96e16f80b480621c/users/1b225b55d18740c2833cade52edc556e/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:47.234778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.236689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.237069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.265201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b47f546baf7341da9abda1a754006d1a', 'request_id': 'req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.268711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:47.270593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:47.355069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 917/1850] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:47.369230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:47.372036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:47.381340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b47f546baf7341da9abda1a754006d1a', 'request_id': 'req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.389925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b47f546baf7341da9abda1a754006d1a', 'request_id': 'req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.407671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] The token cache is being invalidate because project b47f546baf7341da9abda1a754006d1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:47.407973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.408726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.410678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 934/1851] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:46 2022] DELETE /identity/v3/projects/b47f546baf7341da9abda1a754006d1a => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:47.415571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:47.418090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:47.443356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e583f-70a8-48ee-b7ae-6b9f2fbb20e9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:47.453167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:47.453559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c52a6e-2226-4a67-b04c-6ad78801e9da None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:47.455288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.455924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.456398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] PATH_INFO: `/v3/users/fb9c206b72f946c1a35df8153edff24f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.458211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb9c206b72f946c1a35df8153edff24f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:47.458515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:47.459526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.461861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:47.474684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.475071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.477889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:47.507404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:47.509260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:47.530101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 918/1852] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:47.758704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:47.761465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:47.762414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb9c206b72f946c1a35df8153edff24f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.774059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb9c206b72f946c1a35df8153edff24f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.775373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb9c206b72f946c1a35df8153edff24f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.776026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb9c206b72f946c1a35df8153edff24f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.791266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c40a7-deeb-41bd-ba2e-b9b502f63fd2 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:47.798862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:47.799565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:47.800402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:47.800402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] PATH_INFO: `/v3/projects/d7e8f24d91f94c698ae6018210deea1d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:47.802578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e8f24d91f94c698ae6018210deea1d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:47.805353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:47.808857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb9c206b72f946c1a35df8153edff24f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.914691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 935/1853] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:47 2022] DELETE /identity/v3/users/fb9c206b72f946c1a35df8153edff24f => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:47.970584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e8f24d91f94c698ae6018210deea1d', 'request_id': 'req-6cf7e41c-f449-4565-83e0-3913dc22b57b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.982857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e8f24d91f94c698ae6018210deea1d', 'request_id': 'req-6cf7e41c-f449-4565-83e0-3913dc22b57b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:47.992561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e8f24d91f94c698ae6018210deea1d', 'request_id': 'req-6cf7e41c-f449-4565-83e0-3913dc22b57b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.105681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] The token cache is being invalidate because project d7e8f24d91f94c698ae6018210deea1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:48.105681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.105681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.105681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 919/1854] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:47 2022] DELETE /identity/v3/projects/d7e8f24d91f94c698ae6018210deea1d => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:48.109642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:48.111942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:48.197628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e537c7-1b80-4101-bc4d-f1432d7d9fbc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:48.276188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:48.276924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:48.277214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:48.277494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] PATH_INFO: `/v3/users/2cd06e7233d74775abb4932266c8b268` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:48.279888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd06e7233d74775abb4932266c8b268)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:48.282550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:48.296559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:48.474533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:48.476168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:48.491597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7e41c-f449-4565-83e0-3913dc22b57b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:48.504757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:48.504757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:48.504757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:48.504757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] PATH_INFO: `/v3/projects/36cc862bdd514b668ea3e11a9ff02b2f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:48.505480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36cc862bdd514b668ea3e11a9ff02b2f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:48.505480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:48.559362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd06e7233d74775abb4932266c8b268'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.566499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd06e7233d74775abb4932266c8b268'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.567212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd06e7233d74775abb4932266c8b268'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.567830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd06e7233d74775abb4932266c8b268'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.580601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6ba88fe-9b8f-4a3e-a086-2aa16781cb7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd06e7233d74775abb4932266c8b268'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.653024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36cc862bdd514b668ea3e11a9ff02b2f', 'request_id': 'req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.668402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 936/1855] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:48 2022] DELETE /identity/v3/users/2cd06e7233d74775abb4932266c8b268 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:48.673316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36cc862bdd514b668ea3e11a9ff02b2f', 'request_id': 'req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.678998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36cc862bdd514b668ea3e11a9ff02b2f', 'request_id': 'req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.832777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] The token cache is being invalidate because project 36cc862bdd514b668ea3e11a9ff02b2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:48.833081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.835113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6ea0fd55-4d19-446e-bf1f-597d0db6a4b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:48.837263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 920/1856] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:48 2022] DELETE /identity/v3/projects/36cc862bdd514b668ea3e11a9ff02b2f => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:48.844126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:48.844660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:48.845086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:48.871732 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:48.872637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:48.982884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 937/1857] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:48.990465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:48.990760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:48.991044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.020784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:49.022172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:49.036376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 921/1858] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:49.040889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.052811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5786d-50b4-4382-821e-d6f3818e3a8d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.058713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.061355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.061355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.061355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.062068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:11:49.062475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.064305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.077075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 938/1859] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:11:49.081298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.090589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b34a5b-5889-485a-bd60-2a566afea64d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.100215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.100951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.101246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.101524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.102332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.104479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.246963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 922/1860] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:49 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:49.251759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.263116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b73a71-0f24-49a6-b28a-b758ab226e4e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.331482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.332287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.332705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.332929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.333518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.335565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.394829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.396708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:49.410480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1133db69-f25a-4f72-98d4-9a5673606261 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.416927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.418154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.418655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.419146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] PATH_INFO: `/v3/projects/ee9ba3eba02e493eb3058a1eb038fab9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.420609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee9ba3eba02e493eb3058a1eb038fab9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.427516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.463219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 939/1861] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:49 2022] POST /identity/v3/users => generated 291 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:49.464504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.475900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b1e82-f349-4116-9d91-86e9ac89a823 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.481820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.487030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.487210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.487546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.488316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.490164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.505406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 940/1862] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:49.507472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.520421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea6562-7b18-427c-bb47-6f4e24042d95 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.525174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.530473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.530473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.530473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.530473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.530473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.544228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 941/1863] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:49.545092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:49.572825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3932438c-37be-4bfc-b4a1-a9ff5c36ca33 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:49.579389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43f9c1-6e7f-49da-8c1d-d8e24abf3f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:49.580186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43f9c1-6e7f-49da-8c1d-d8e24abf3f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.580725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43f9c1-6e7f-49da-8c1d-d8e24abf3f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.581084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43f9c1-6e7f-49da-8c1d-d8e24abf3f68 admin admin] PATH_INFO: `/v3/projects/acdb64a742874d56a1984a7f149becd5/users/b056e79efcc049cc8653f8f23df040a1/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.598599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43f9c1-6e7f-49da-8c1d-d8e24abf3f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acdb64a742874d56a1984a7f149becd5, user_id=b056e79efcc049cc8653f8f23df040a1, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:49.601014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43f9c1-6e7f-49da-8c1d-d8e24abf3f68 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:49.625535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee9ba3eba02e493eb3058a1eb038fab9', 'request_id': 'req-766554f2-b021-43f7-ad20-72b479292e1e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:49.633207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee9ba3eba02e493eb3058a1eb038fab9', 'request_id': 'req-766554f2-b021-43f7-ad20-72b479292e1e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:49.640971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee9ba3eba02e493eb3058a1eb038fab9', 'request_id': 'req-766554f2-b021-43f7-ad20-72b479292e1e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:49.794178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] The token cache is being invalidate because project ee9ba3eba02e493eb3058a1eb038fab9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:49.794659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:49.795493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-766554f2-b021-43f7-ad20-72b479292e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:49.797245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 923/1864] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:49 2022] DELETE /identity/v3/projects/ee9ba3eba02e493eb3058a1eb038fab9 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:49.799180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab7964-2b18-478e-97db-cb1b93577ea0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.799684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab7964-2b18-478e-97db-cb1b93577ea0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.799967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab7964-2b18-478e-97db-cb1b93577ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.802489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 942/1865] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:49 2022] PUT /identity/v3/projects/acdb64a742874d56a1984a7f149becd5/users/b056e79efcc049cc8653f8f23df040a1/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:49.814057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f777fb-48c7-45cd-a8c0-5008ddf54784 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.814057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f777fb-48c7-45cd-a8c0-5008ddf54784 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.814057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f777fb-48c7-45cd-a8c0-5008ddf54784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:49.839998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-83f777fb-48c7-45cd-a8c0-5008ddf54784 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:49.841912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-83f777fb-48c7-45cd-a8c0-5008ddf54784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:49.884309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:49.888618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fdab7964-2b18-478e-97db-cb1b93577ea0 None None] MFA Rules not processed for user `1b225b55d18740c2833cade52edc556e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:49.890070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdab7964-2b18-478e-97db-cb1b93577ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:49.965689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 943/1866] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:49.972114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:49.972373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:49.972667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.010687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] MFA Rules not processed for user `b056e79efcc049cc8653f8f23df040a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:50.011935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:50.033929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 924/1867] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:50.035511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.035806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.036082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.092124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:50.093504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:50.111783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 925/1868] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:50.115919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.120377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:50.120692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 944/1869] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:50.121774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.139072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fffe1f1-75f2-4841-8677-60cce417aebb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.144058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.144640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.144923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.145197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.145885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:11:50.146190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.147929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.163055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 945/1870] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:11:50.163055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.164200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:50.181417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa53d530-7d4d-44a1-82d0-fe8170a81795 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.202443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.203076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.203359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.203667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.204361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.206519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.234697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e5ec8-b98d-4dde-b5d4-35350f862281 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.240056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.240950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.241406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.242147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.255193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.256700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.267487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 926/1871] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:50.281358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.297617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc37ff4-5908-46ce-b038-04c3abe2cad2 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.303411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.304031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.304572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.305165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.306094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.308394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.400857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 946/1872] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:50 2022] POST /identity/v3/projects => generated 383 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:50.402672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.439672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 927/1873] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:50 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:50.444357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.514250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6303e-8252-419c-82db-1564a10fd384 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.532493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c18f0f7-4123-4738-9c01-5a9e419b6f1c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.537937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.538708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.539211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.539702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.540593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.542827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.583263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.583867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.584150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.584499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.585068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.587181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.670591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 928/1874] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:50 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:50.675658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.687048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9789d9d-6bcf-462f-b33d-ff36e2ae3d31 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.700603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 947/1875] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:50 2022] POST /identity/v3/users => generated 298 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:50.704714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.707671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.707671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.707860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.708552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.709827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.710383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.724536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 929/1876] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:50.724727 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f549d2e-036d-44e1-9992-5295a54d9a2e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.728288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.730876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.732120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.732636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.733581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.734536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.736337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.738050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e52a74b-895f-4d8e-8b2c-c2c29b49c978 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.742934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.743634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.744518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.744780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.745606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.748512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.755667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 948/1877] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:50.759909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.764362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 930/1878] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:50.766406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.774348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38db3cf4-2f0b-40df-808e-f7b851eee5c1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.777570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badfb59e-0296-422b-b859-6bc1d867b9d4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:50.783231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.787958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96713322-2ba9-4c3e-8dba-b2c1536113f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:50.791349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.791349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.791349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] PATH_INFO: `/v3/projects/ea0ecf66d13341e1921b6f1e7b74e1a0/users/50f5408288304590a338f4f10e6b0917/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.791658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96713322-2ba9-4c3e-8dba-b2c1536113f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.791658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96713322-2ba9-4c3e-8dba-b2c1536113f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.792493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96713322-2ba9-4c3e-8dba-b2c1536113f2 admin admin] PATH_INFO: `/v3/projects/4e148d4647f44f28a8060a8c0c6c4a7d/users/65d44a14dc81450dbd5d2a790c940180/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:50.829474 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea0ecf66d13341e1921b6f1e7b74e1a0, user_id=50f5408288304590a338f4f10e6b0917, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.831186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96713322-2ba9-4c3e-8dba-b2c1536113f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e148d4647f44f28a8060a8c0c6c4a7d, user_id=65d44a14dc81450dbd5d2a790c940180, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:50.832616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.834615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96713322-2ba9-4c3e-8dba-b2c1536113f2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:50.954239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 949/1879] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:50 2022] PUT /identity/v3/projects/ea0ecf66d13341e1921b6f1e7b74e1a0/users/50f5408288304590a338f4f10e6b0917/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:50.958832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:50.968935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 931/1880] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:50 2022] PUT /identity/v3/projects/4e148d4647f44f28a8060a8c0c6c4a7d/users/65d44a14dc81450dbd5d2a790c940180/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:50.970484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:50.971342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:50.971738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.029464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] MFA Rules not processed for user `65d44a14dc81450dbd5d2a790c940180`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:51.031491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:51.086565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d23d5-0800-45b3-9da7-b8aede937496 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.163700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 932/1881] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:51.168206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.234536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.239381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.239381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.239381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.239381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.239381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.252415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 950/1882] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:51.256796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.271494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a12f9-c28b-445b-b6dc-f0c7ab4ff66f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.277825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6666263c-b571-4c11-9ec0-a05c7ebc0c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.279695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666263c-b571-4c11-9ec0-a05c7ebc0c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.279695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666263c-b571-4c11-9ec0-a05c7ebc0c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.279864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6666263c-b571-4c11-9ec0-a05c7ebc0c41 admin admin] PATH_INFO: `/v3/projects/ea0ecf66d13341e1921b6f1e7b74e1a0/users/50f5408288304590a338f4f10e6b0917/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.282020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666263c-b571-4c11-9ec0-a05c7ebc0c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea0ecf66d13341e1921b6f1e7b74e1a0, user_id=50f5408288304590a338f4f10e6b0917, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.285034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6666263c-b571-4c11-9ec0-a05c7ebc0c41 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.285919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a2449-cdba-4e6a-90ca-080851d704a7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.291860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.292795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.293114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.293548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.295680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.298074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.308894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 951/1883] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:51 2022] PUT /identity/v3/projects/ea0ecf66d13341e1921b6f1e7b74e1a0/users/50f5408288304590a338f4f10e6b0917/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:11:51.311783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.312075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.312351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.347781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] MFA Rules not processed for user `50f5408288304590a338f4f10e6b0917`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:51.349194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:51.424956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 952/1884] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:51.441578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.478015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 933/1885] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:51 2022] POST /identity/v3/projects => generated 379 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:51.478966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.505175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef5f16-98e2-4b62-86d3-7fc43c7c55d6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.555730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92730262-b5c6-40c3-8112-4e3242bf7086 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.563322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.563955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.564294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.564716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.565364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.565622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.566368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.566756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.567127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.567844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.638885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.640397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.650238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 953/1886] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:51.729452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 934/1887] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:51 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:51.733608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.746227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421f22a7-4de1-4693-98fc-fade132e97b9 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.751708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.752322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.752598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.752868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.753560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.755190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.765768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.766864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 954/1888] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:11:51.770766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.776291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c638c-9d4c-42a4-a94d-b0b47c445d7e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.780373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e583a-399a-4ea0-83db-08412a46b31c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.780976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d09c5-ee47-485d-afd0-f6ddf07a83bc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.781087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e583a-399a-4ea0-83db-08412a46b31c service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.781266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e583a-399a-4ea0-83db-08412a46b31c service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.781552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e583a-399a-4ea0-83db-08412a46b31c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.785818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.792234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.792234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.792234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.792234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.792234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.805231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 955/1889] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:51.806415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:51.817876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b9de7-1347-499d-bd20-4fd67aeb3922 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:51.823156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:51.823996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.824277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.824559 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] PATH_INFO: `/v3/projects/bd36dc7f57674251a92ba27a76428e70/users/418f70d4837e4421846feca52a038a84/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:51.843959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd36dc7f57674251a92ba27a76428e70, user_id=418f70d4837e4421846feca52a038a84, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.849799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.870844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e583a-399a-4ea0-83db-08412a46b31c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:51.872947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e583a-399a-4ea0-83db-08412a46b31c service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:51.883165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 935/1890] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:11:51.907130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:51.949248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 956/1891] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:51 2022] PUT /identity/v3/projects/bd36dc7f57674251a92ba27a76428e70/users/418f70d4837e4421846feca52a038a84/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:51.960781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:51.961123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:51.961406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.010638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] MFA Rules not processed for user `418f70d4837e4421846feca52a038a84`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:52.010638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:52.079029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 936/1892] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:52.083602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:52.138997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:52.166590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f399fa-eeeb-4502-9146-8f0eb37f1f2b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:52.171622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db044aa-06e6-48b6-abdd-b1a955701120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:52.172251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db044aa-06e6-48b6-abdd-b1a955701120 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:52.172514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db044aa-06e6-48b6-abdd-b1a955701120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:52.173055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db044aa-06e6-48b6-abdd-b1a955701120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.174374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db044aa-06e6-48b6-abdd-b1a955701120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:52.176391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db044aa-06e6-48b6-abdd-b1a955701120 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:52.279383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 957/1893] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:11:52 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:11:52.283789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:52.291991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:52.293461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:52.293814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.310036 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819a5cc-877b-4d66-ae6d-3d74a1196787 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:52.326334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] MFA Rules not processed for user `1b225b55d18740c2833cade52edc556e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:52.327568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:52.400947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:52.401624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:52.402121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:52.402580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.403888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:52.408334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:52.449202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 958/1894] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:52.672216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 937/1895] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:11:52 2022] POST /identity/v3/users => generated 295 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:11:52.676841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:52.687769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aeccca-2e6d-4b9a-b46f-b1358bc527fd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:52.692347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:52.693565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:52.693831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:52.698088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.698088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:52.698088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:52.712879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 959/1896] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:52.717711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:52.733517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90c7fa3-2d07-4056-bd76-b9b793598f3e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:52.739464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:52.740157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:52.740437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:52.740715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] PATH_INFO: `/v3/projects/0d25acce99704381bec40b75f4077b45/users/645f69210fca4cd5a30e1ad25d6d7e4a/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.763599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d25acce99704381bec40b75f4077b45, user_id=645f69210fca4cd5a30e1ad25d6d7e4a, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:52.766317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:52.863843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 938/1897] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:52 2022] PUT /identity/v3/projects/0d25acce99704381bec40b75f4077b45/users/645f69210fca4cd5a30e1ad25d6d7e4a/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:52.865045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:52.898467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893cc95-efa6-4434-bc0a-bb4d2d4c5b7c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:52.980729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd137cac-51fb-4b7d-bb12-c8ff22fcf947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:52.981956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd137cac-51fb-4b7d-bb12-c8ff22fcf947 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:52.982317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd137cac-51fb-4b7d-bb12-c8ff22fcf947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:52.982590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd137cac-51fb-4b7d-bb12-c8ff22fcf947 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:52.983410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd137cac-51fb-4b7d-bb12-c8ff22fcf947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:52.986619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd137cac-51fb-4b7d-bb12-c8ff22fcf947 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:53.003101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 960/1898] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:11:53.004859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:53.017684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc27c4e-da4f-407b-ba34-c2182c13f5d6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:53.024100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:53.024803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:53.025098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:53.025386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] PATH_INFO: `/v3/projects/0d25acce99704381bec40b75f4077b45/users/645f69210fca4cd5a30e1ad25d6d7e4a/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:53.028577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d25acce99704381bec40b75f4077b45, user_id=645f69210fca4cd5a30e1ad25d6d7e4a, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:53.032233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:53.046438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 939/1899] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:11:53 2022] PUT /identity/v3/projects/0d25acce99704381bec40b75f4077b45/users/645f69210fca4cd5a30e1ad25d6d7e4a/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:53.058362 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:53.059135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:53.060266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:53.094121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] MFA Rules not processed for user `645f69210fca4cd5a30e1ad25d6d7e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:53.097134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:53.203880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 961/1900] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:53.216942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:53.293579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a0525-5dcb-47cd-b654-036bf6bf44c1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:53.298583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:53.299231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:53.299552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:53.299830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:53.311264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:53.313088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:53.340936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 940/1901] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:53 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:53.541338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:53.565259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08e6588-0aaf-474d-8415-a2fdd7aede59 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:53.570535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967bac6d-48cb-4eab-8650-644bf2ff046a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:53.571266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bac6d-48cb-4eab-8650-644bf2ff046a service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:53.571731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bac6d-48cb-4eab-8650-644bf2ff046a service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:53.572030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bac6d-48cb-4eab-8650-644bf2ff046a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:53.649986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bac6d-48cb-4eab-8650-644bf2ff046a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:53.651017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967bac6d-48cb-4eab-8650-644bf2ff046a service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:53.663726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 962/1902] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:53.877885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:54.023759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:56.341228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:56.424147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d499e92-3b24-49ce-9d4b-c6844709dcc4 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:56.429053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:56.429995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:56.430281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:56.431269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] PATH_INFO: `/v3/users/b056e79efcc049cc8653f8f23df040a1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:56.432659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b056e79efcc049cc8653f8f23df040a1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:56.436015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:56.550975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:56.741102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:56.765124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b056e79efcc049cc8653f8f23df040a1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:56.774266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b056e79efcc049cc8653f8f23df040a1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:56.848931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b056e79efcc049cc8653f8f23df040a1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:56.849643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b056e79efcc049cc8653f8f23df040a1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:56.877965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b056e79efcc049cc8653f8f23df040a1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:56.893778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 941/1903] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:56 2022] DELETE /identity/v3/users/b056e79efcc049cc8653f8f23df040a1 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:56.898567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:56.898845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:56.902616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:56.930638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:56.932278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:57.032907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 963/1904] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:57.038587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:57.040146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:57.104795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fe4a03-32d5-4143-8483-2a03d43b5227 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:57.108867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:57.109502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:57.109819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:57.111570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:57.124191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:57.125867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:57.136254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 942/1905] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:57.707062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:57.708726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:57.729239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6c238-20a1-474b-8419-282966aa9f4f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:57.735536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:57.736093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:57.736378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:57.736655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] PATH_INFO: `/v3/projects/acdb64a742874d56a1984a7f149becd5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:57.738007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=acdb64a742874d56a1984a7f149becd5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:57.740017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:57.857008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acdb64a742874d56a1984a7f149becd5', 'request_id': 'req-b25025f8-863c-40a5-b763-97707cd899dd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:57.966524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acdb64a742874d56a1984a7f149becd5', 'request_id': 'req-b25025f8-863c-40a5-b763-97707cd899dd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:57.973783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acdb64a742874d56a1984a7f149becd5', 'request_id': 'req-b25025f8-863c-40a5-b763-97707cd899dd'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:57.985496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] The token cache is being invalidate because project acdb64a742874d56a1984a7f149becd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:57.985834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:57.986556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:57.988198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 964/1906] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:57 2022] DELETE /identity/v3/projects/acdb64a742874d56a1984a7f149becd5 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:57.992503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:58.003432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd1a8-720d-4387-8b84-d60923fbf4c1 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:58.074344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:58.078618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:58.078618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:58.078618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] PATH_INFO: `/v3/users/50f5408288304590a338f4f10e6b0917` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:58.078618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f5408288304590a338f4f10e6b0917)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:58.079149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:58.280262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f5408288304590a338f4f10e6b0917'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.289529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f5408288304590a338f4f10e6b0917'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.376517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f5408288304590a338f4f10e6b0917'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.376517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f5408288304590a338f4f10e6b0917'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.390031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-82c1b9e9-ed10-4823-8ba7-34fdf169e11b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f5408288304590a338f4f10e6b0917'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.411090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 943/1907] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:57 2022] DELETE /identity/v3/users/50f5408288304590a338f4f10e6b0917 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:58.548989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:58.550571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:58.565181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25025f8-863c-40a5-b763-97707cd899dd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:58.621141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:58.632348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:58.633140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:58.633497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:58.633880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] PATH_INFO: `/v3/users/65d44a14dc81450dbd5d2a790c940180` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:58.635356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d44a14dc81450dbd5d2a790c940180)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:58.638408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:58.874206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d44a14dc81450dbd5d2a790c940180'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.882123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d44a14dc81450dbd5d2a790c940180'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.882874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d44a14dc81450dbd5d2a790c940180'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.883491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d44a14dc81450dbd5d2a790c940180'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:58.900224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d44a14dc81450dbd5d2a790c940180'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:59.006481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:59.045989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 965/1908] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:11:58 2022] DELETE /identity/v3/users/65d44a14dc81450dbd5d2a790c940180 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:59.050760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:59.051783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:59.052316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:59.080309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:59.081708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:59.148229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 944/1909] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:11:59.159935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:59.161433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:59.243358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7692499c-1c78-4650-84dc-d27945a5cace admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:59.247710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05ffac-3f0e-4097-a513-835738d08a1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:59.248403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05ffac-3f0e-4097-a513-835738d08a1f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:59.248817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05ffac-3f0e-4097-a513-835738d08a1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:59.249095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05ffac-3f0e-4097-a513-835738d08a1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:59.259851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05ffac-3f0e-4097-a513-835738d08a1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:59.261431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05ffac-3f0e-4097-a513-835738d08a1f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:59.271674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 966/1910] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:11:59.414626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:59.417798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:59.444699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b92e4-4de8-4e8a-9fb1-1b057933e0e1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:59.451477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:59.452142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:59.452426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:59.452711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] PATH_INFO: `/v3/projects/ea0ecf66d13341e1921b6f1e7b74e1a0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:59.454037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea0ecf66d13341e1921b6f1e7b74e1a0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:59.455967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:59.596458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea0ecf66d13341e1921b6f1e7b74e1a0', 'request_id': 'req-d16dc733-d78d-41ff-8679-897db6f4527a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:59.605502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea0ecf66d13341e1921b6f1e7b74e1a0', 'request_id': 'req-d16dc733-d78d-41ff-8679-897db6f4527a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:59.613418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea0ecf66d13341e1921b6f1e7b74e1a0', 'request_id': 'req-d16dc733-d78d-41ff-8679-897db6f4527a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:59.724630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] The token cache is being invalidate because project ea0ecf66d13341e1921b6f1e7b74e1a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:11:59.725188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:59.725990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:11:59.727707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 945/1911] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:59 2022] DELETE /identity/v3/projects/ea0ecf66d13341e1921b6f1e7b74e1a0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:11:59.733535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fad52-8900-4fad-b32d-bad8c909c975 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:59.733535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fad52-8900-4fad-b32d-bad8c909c975 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:59.733535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877fad52-8900-4fad-b32d-bad8c909c975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:59.781073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-877fad52-8900-4fad-b32d-bad8c909c975 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:59.782358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-877fad52-8900-4fad-b32d-bad8c909c975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:59.881198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:59.882376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:59.903106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 967/1912] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:59.909722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:11:59.910030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:59.910030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:59.910030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:59.910264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16dc733-d78d-41ff-8679-897db6f4527a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:11:59.915106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:11:59.916009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:11:59.916978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:11:59.917358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] PATH_INFO: `/v3/projects/4e148d4647f44f28a8060a8c0c6c4a7d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:11:59.919869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e148d4647f44f28a8060a8c0c6c4a7d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:11:59.922928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:11:59.940266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:11:59.942217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:11:59.969968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 968/1913] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:11:59.970974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:11:59.997631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4208be9-f3cb-45ad-978b-1a14a2477421 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.004175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.004873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.005218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.005561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.006542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:00.006980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.008891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.025978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 969/1914] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:00.030821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:00.073547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e148d4647f44f28a8060a8c0c6c4a7d', 'request_id': 'req-305da6a8-9cdd-4f75-a442-01103950f96c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.075519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7c0348-9824-4336-8c4f-27b958a504e2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.084533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e148d4647f44f28a8060a8c0c6c4a7d', 'request_id': 'req-305da6a8-9cdd-4f75-a442-01103950f96c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.107277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e148d4647f44f28a8060a8c0c6c4a7d', 'request_id': 'req-305da6a8-9cdd-4f75-a442-01103950f96c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.192622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.193644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.194156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.194628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.196100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.198801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.207521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] The token cache is being invalidate because project 4e148d4647f44f28a8060a8c0c6c4a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:00.208039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.214594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.214594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 946/1915] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:11:59 2022] DELETE /identity/v3/projects/4e148d4647f44f28a8060a8c0c6c4a7d => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:00.217993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:00.220628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:00.236090 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305da6a8-9cdd-4f75-a442-01103950f96c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.246129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.246129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.246129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.246129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] PATH_INFO: `/v3/users/418f70d4837e4421846feca52a038a84` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.247262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=418f70d4837e4421846feca52a038a84)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.248387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.326665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 970/1916] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:00 2022] POST /identity/v3/projects => generated 367 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:00.339747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:00.343101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:00.482250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaecb08-73bd-47cf-9197-89bae2298c14 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.489191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.489878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.490550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.490550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.491069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.493951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.564407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '418f70d4837e4421846feca52a038a84'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.579057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '418f70d4837e4421846feca52a038a84'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.601297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 971/1917] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:00 2022] POST /identity/v3/users => generated 284 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:00.605394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:00.616404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb4faf5-6bf3-4e47-86c7-2d4555452f06 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.621381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.622297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.626409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.626409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.626409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.626409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.636789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 972/1918] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:00.640945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:00.651604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8620b34f-cc27-4961-8857-d0ce743d93b2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.657144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.657836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.658144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.658419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.659055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.660663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.672236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 973/1919] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:00.676582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:00.686275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871f20f-425d-4591-8f13-d8bbb0b41188 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:00.690908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:00.691611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.691843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.692100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] PATH_INFO: `/v3/projects/8095bc61e93444e0b1912c431433a242/users/0be006c783a2402bae8b8383d0f7bf33/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.707631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8095bc61e93444e0b1912c431433a242, user_id=0be006c783a2402bae8b8383d0f7bf33, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:00.710353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:00.732595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '418f70d4837e4421846feca52a038a84'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.733400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '418f70d4837e4421846feca52a038a84'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.759331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f1774b8d-ea9a-4766-8fc8-fd9cdc6298ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '418f70d4837e4421846feca52a038a84'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:00.772017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 947/1920] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:00 2022] DELETE /identity/v3/users/418f70d4837e4421846feca52a038a84 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:00.836090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 974/1921] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:00 2022] PUT /identity/v3/projects/8095bc61e93444e0b1912c431433a242/users/0be006c783a2402bae8b8383d0f7bf33/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:00.848029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:00.848968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:00.849263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:00.911787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] MFA Rules not processed for user `0be006c783a2402bae8b8383d0f7bf33`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:00.913251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:01.022920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 948/1922] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:01.023197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.024961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:01.136757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd4e3b-b57a-45e4-bab0-8e29be448846 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.150293 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.151150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.151663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.152185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.157422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.158805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.204277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:01.263790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 975/1923] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:01 2022] POST /identity/v3/projects => generated 365 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:01.266713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.279403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdbb22-87a2-40a5-9c00-1e57b4043947 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.350394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.351516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.351516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.352101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.352951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.355556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.455582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 949/1924] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:01 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:01.456246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.475586 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.477391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:01.479967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b521768-a58a-4434-ace5-70cf76a0dcea admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.492827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.493169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55131fc0-6c44-4f0b-9087-2852a4bc7dd3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.494387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.494903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.495976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.498186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.499784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.500287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.500889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.501401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.501803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] PATH_INFO: `/v3/projects/bd36dc7f57674251a92ba27a76428e70` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.503383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd36dc7f57674251a92ba27a76428e70)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.506628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.523263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 976/1925] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:01.529286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.544489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65395ca-9def-486a-9c15-6b70149f8a29 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.563620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.564425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.564785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.565144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] PATH_INFO: `/v3/projects/7216364fa6b444478d94f860a518a121/users/f2f9ca50c8f44e08a0d8bc69c58a6a41/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.584581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7216364fa6b444478d94f860a518a121, user_id=f2f9ca50c8f44e08a0d8bc69c58a6a41, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.589562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.667771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 977/1926] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:01 2022] PUT /identity/v3/projects/7216364fa6b444478d94f860a518a121/users/f2f9ca50c8f44e08a0d8bc69c58a6a41/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:01.673083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.682142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd36dc7f57674251a92ba27a76428e70', 'request_id': 'req-ba25c7f8-b77e-4032-a355-744d133d8010'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:01.690873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd36dc7f57674251a92ba27a76428e70', 'request_id': 'req-ba25c7f8-b77e-4032-a355-744d133d8010'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:01.697640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd36dc7f57674251a92ba27a76428e70', 'request_id': 'req-ba25c7f8-b77e-4032-a355-744d133d8010'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:01.744700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b9d4c3-7090-42ff-8130-1196c3f841c4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.773906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] The token cache is being invalidate because project bd36dc7f57674251a92ba27a76428e70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:01.774418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:01.775074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:01.779851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 950/1927] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:01 2022] DELETE /identity/v3/projects/bd36dc7f57674251a92ba27a76428e70 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:01.780699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.782068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:01.802805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.803425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.803711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.804022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.804667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.806331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.817375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 978/1928] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:01 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:01.821464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.823252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:01.835938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2ade2e-4db2-4a43-80ed-c51b2b528744 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.840658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5e5b32-9d88-4df1-a1a3-cc45108a3918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.841368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5b32-9d88-4df1-a1a3-cc45108a3918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.841712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5b32-9d88-4df1-a1a3-cc45108a3918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.842029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e5b32-9d88-4df1-a1a3-cc45108a3918 admin admin] PATH_INFO: `/v3/projects/7216364fa6b444478d94f860a518a121/users/f2f9ca50c8f44e08a0d8bc69c58a6a41/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.843958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5b32-9d88-4df1-a1a3-cc45108a3918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7216364fa6b444478d94f860a518a121, user_id=f2f9ca50c8f44e08a0d8bc69c58a6a41, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.846670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5e5b32-9d88-4df1-a1a3-cc45108a3918 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.847669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25c7f8-b77e-4032-a355-744d133d8010 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:01.852442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:01.853117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.853449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.853679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] PATH_INFO: `/v3/users/645f69210fca4cd5a30e1ad25d6d7e4a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.854896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] RBAC: Authorizing `identity:delete_user(user_id=645f69210fca4cd5a30e1ad25d6d7e4a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:01.857331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:01.857673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 979/1929] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:01 2022] PUT /identity/v3/projects/7216364fa6b444478d94f860a518a121/users/f2f9ca50c8f44e08a0d8bc69c58a6a41/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:01.867921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:01.868214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:01.868486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:01.897200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] MFA Rules not processed for user `f2f9ca50c8f44e08a0d8bc69c58a6a41`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:01.898508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:01.967097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 980/1930] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:01.978456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:01.979857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:02.042902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92741b43-c2be-4f81-8674-0a0acd95d99b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:02.047701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:02.048322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:02.048602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:02.048876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:02.058180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:02.059730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:02.069667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 981/1931] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:02.072954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '645f69210fca4cd5a30e1ad25d6d7e4a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:02.080294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '645f69210fca4cd5a30e1ad25d6d7e4a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:02.191241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '645f69210fca4cd5a30e1ad25d6d7e4a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:02.191975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '645f69210fca4cd5a30e1ad25d6d7e4a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:02.208242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '645f69210fca4cd5a30e1ad25d6d7e4a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:02.220596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 951/1932] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:01 2022] DELETE /identity/v3/users/645f69210fca4cd5a30e1ad25d6d7e4a => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:02.383980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:02.394494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f03e3-5a77-432b-8343-b63756defd7c service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:02.397644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:02.456393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:02.457314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:02.457768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:02.458191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:02.460416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:02.557871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:02.560682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025c0de-c96a-49b6-867d-99bacc73a63a service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:02.572320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 982/1933] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:02.765309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:02.766780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:02.849384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320d0552-9264-4358-8ec0-7f3e8b31bbeb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:02.856296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:02.857310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:02.857310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:02.857585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] PATH_INFO: `/v3/projects/0d25acce99704381bec40b75f4077b45` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:02.858905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d25acce99704381bec40b75f4077b45)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:02.860704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:02.990041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d25acce99704381bec40b75f4077b45', 'request_id': 'req-b5858b59-b9ac-4b57-9409-887e9f610950'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:02.997277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d25acce99704381bec40b75f4077b45', 'request_id': 'req-b5858b59-b9ac-4b57-9409-887e9f610950'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:03.003448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d25acce99704381bec40b75f4077b45', 'request_id': 'req-b5858b59-b9ac-4b57-9409-887e9f610950'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:03.109563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] The token cache is being invalidate because project 0d25acce99704381bec40b75f4077b45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:03.109563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:03.109563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b5858b59-b9ac-4b57-9409-887e9f610950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:03.109563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 952/1934] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:02 2022] DELETE /identity/v3/projects/0d25acce99704381bec40b75f4077b45 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:03.110181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.110181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.110181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.129940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:03.131264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:03.198604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 983/1935] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:03.208751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.209034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.209361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.233220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:03.234497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:03.243563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 953/1936] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:03.251600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:03.261798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d43038-dce2-4e1a-965a-cbec5258a8bb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:03.270311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:03.271099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.271392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.272215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.273092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:03.273409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:03.275134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:03.287163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:03.291280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 984/1937] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:03.306623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:03.317653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce7b26-208e-4b35-aa9a-3f612073d9e0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:03.323599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:03.324217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.325126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.325126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.325489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:03.327514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:03.476782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 954/1938] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:03 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:03.480600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:03.498879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54576e8-2ee0-45d5-80f8-d98a51b26107 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:03.562230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:03.562859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.570964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.570964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.570964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:03.570964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:03.688521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 985/1939] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:03 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:03.692945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:03.705669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da821-c363-4e65-b826-a8f7bae760f5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:03.720305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:03.720305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.720305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.720305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.720654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:03.720654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:03.732492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 955/1940] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:03.733954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:03.745150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb2712-c127-4672-90c5-78a54335f758 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:03.751155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34bf12b-3ba8-440b-844a-20a3d0eb969c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:03.757355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf12b-3ba8-440b-844a-20a3d0eb969c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.757355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf12b-3ba8-440b-844a-20a3d0eb969c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.757355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34bf12b-3ba8-440b-844a-20a3d0eb969c admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.757355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf12b-3ba8-440b-844a-20a3d0eb969c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:03.757355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34bf12b-3ba8-440b-844a-20a3d0eb969c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:03.770244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 986/1941] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:03.775015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:03.788196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2ec5f-1fdb-4f19-b572-02bc59f5bd2a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:03.793260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:03.794544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.794828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.795108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] PATH_INFO: `/v3/projects/485122ccfa3e46f8a024383a0603aa7b/users/347620eeff8344bb872d699759ef3eab/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.811231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] RBAC: Authorizing `identity:create_grant(project_id=485122ccfa3e46f8a024383a0603aa7b, user_id=347620eeff8344bb872d699759ef3eab, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:03.814677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:03.898256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 956/1942] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:03 2022] PUT /identity/v3/projects/485122ccfa3e46f8a024383a0603aa7b/users/347620eeff8344bb872d699759ef3eab/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:03.911210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:03.911510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:03.911814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:03.947361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] MFA Rules not processed for user `347620eeff8344bb872d699759ef3eab`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:03.949381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:04.053137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 987/1943] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:04.066412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:04.084134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:04.179414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88e6d0-f7bf-4d4a-bd3f-885944c3602d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:04.183286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf1e60-79ad-451e-a73a-b2207fc1a87f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:04.183997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf1e60-79ad-451e-a73a-b2207fc1a87f service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:04.184708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf1e60-79ad-451e-a73a-b2207fc1a87f service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:04.185093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf1e60-79ad-451e-a73a-b2207fc1a87f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:04.195901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf1e60-79ad-451e-a73a-b2207fc1a87f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:04.197878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf1e60-79ad-451e-a73a-b2207fc1a87f service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:04.209252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 957/1944] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:04.231582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:04.312442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7296df3-50ca-4979-b069-6f6863615e23 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:04.319051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:04.319791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:04.320327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:04.320682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] PATH_INFO: `/v3/users/347620eeff8344bb872d699759ef3eab` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:04.322211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=347620eeff8344bb872d699759ef3eab)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:04.324507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:04.604645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347620eeff8344bb872d699759ef3eab'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:04.613484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347620eeff8344bb872d699759ef3eab'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:04.614223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347620eeff8344bb872d699759ef3eab'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:04.614854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347620eeff8344bb872d699759ef3eab'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:04.629697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347620eeff8344bb872d699759ef3eab'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:04.718222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 988/1945] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:04 2022] DELETE /identity/v3/users/347620eeff8344bb872d699759ef3eab => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:04.723149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:04.723723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:04.723723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:04.745116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:04.747494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:04.817050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 958/1946] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:04.829530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:04.831127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:04.893140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135ce30-6c5b-4f92-9171-a37614988f0e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:04.898424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853d750-23d3-4d45-ab76-039ce4c0fe9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:04.899046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853d750-23d3-4d45-ab76-039ce4c0fe9d service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:04.899330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853d750-23d3-4d45-ab76-039ce4c0fe9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:04.899607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853d750-23d3-4d45-ab76-039ce4c0fe9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:04.915845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853d750-23d3-4d45-ab76-039ce4c0fe9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:04.915845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f853d750-23d3-4d45-ab76-039ce4c0fe9d service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:04.923089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 989/1947] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:04.940527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:05.752869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:05.760934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:05.767466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5cd71-aebf-4c8c-99d3-f00565d43c15 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:05.772685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:05.773331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:05.773909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:05.774189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] PATH_INFO: `/v3/projects/485122ccfa3e46f8a024383a0603aa7b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:05.776369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=485122ccfa3e46f8a024383a0603aa7b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:05.780293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:05.890028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '485122ccfa3e46f8a024383a0603aa7b', 'request_id': 'req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:05.896184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '485122ccfa3e46f8a024383a0603aa7b', 'request_id': 'req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:05.902072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '485122ccfa3e46f8a024383a0603aa7b', 'request_id': 'req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:05.977199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] The token cache is being invalidate because project 485122ccfa3e46f8a024383a0603aa7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:05.977489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:05.978351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0ed8d19e-b3cb-4b57-83b8-16ffa1fd3770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:05.979888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 959/1948] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:05 2022] DELETE /identity/v3/projects/485122ccfa3e46f8a024383a0603aa7b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:05.984893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:05.985404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:05.985502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.006013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:06.007361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:06.072046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 990/1949] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:06.080482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.085468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.085468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.100599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:06.101848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:06.110185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 960/1950] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:06.114844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.125706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc885fa-2cbc-4116-8e57-7158e28599ce None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.130240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.130863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.131145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.131424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.132076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:06.132383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.134158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.150610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 991/1951] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:06.151704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.162046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc7d11-649e-4820-a0ae-098fb701baf5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.167221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.167915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.171653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.171653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.171653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.182890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.291591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 961/1952] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:06 2022] POST /identity/v3/projects => generated 361 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:06.296820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.315812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c99c-e20e-4142-8892-57b1f4ada1d0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.364411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:06.380680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.381423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.381812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.382162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.382868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.384973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.471662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 992/1953] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:06 2022] POST /identity/v3/users => generated 281 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:06.475727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.487082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291fbc21-f335-4c48-890b-af6e07fb62dc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.492374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.492998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.493278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.493557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.494280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.496145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.508189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 962/1954] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:06.512457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.532738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9e6df4-9d5e-4e05-a22c-9f2dea97b61d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.532738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39b717-4417-4102-9fad-61391382ea70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.532738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39b717-4417-4102-9fad-61391382ea70 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.533625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39b717-4417-4102-9fad-61391382ea70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.534119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39b717-4417-4102-9fad-61391382ea70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.534981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39b717-4417-4102-9fad-61391382ea70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.537002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39b717-4417-4102-9fad-61391382ea70 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.549338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 993/1955] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:06.553486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.564401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce77600-41da-4b31-bece-dcc0fedbb60d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.569312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.570014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.570304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.570612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] PATH_INFO: `/v3/projects/28dbfbf36541410baae15996643dd687/users/97adac4405544fdabfceb9a56a66e36f/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.587654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28dbfbf36541410baae15996643dd687, user_id=97adac4405544fdabfceb9a56a66e36f, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.591285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.709579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 963/1956] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:06 2022] PUT /identity/v3/projects/28dbfbf36541410baae15996643dd687/users/97adac4405544fdabfceb9a56a66e36f/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:06.722160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.722501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.722856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.753413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] MFA Rules not processed for user `97adac4405544fdabfceb9a56a66e36f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:06.756202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:06.823656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 994/1957] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:06.861423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.863141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:06.919751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88756ee-8103-470b-88cc-e9179a4df795 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:06.923654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f429d-fbb2-4cdd-8fc6-42241174ad4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:06.924351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f429d-fbb2-4cdd-8fc6-42241174ad4f service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:06.924637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f429d-fbb2-4cdd-8fc6-42241174ad4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:06.924915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f429d-fbb2-4cdd-8fc6-42241174ad4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:06.934409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f429d-fbb2-4cdd-8fc6-42241174ad4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:06.935800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f429d-fbb2-4cdd-8fc6-42241174ad4f service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:06.945960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 964/1958] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:06.962113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:06.971914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:07.061921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c039f21-cb69-478a-9f45-45ce31d15dc7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:07.070360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:07.071200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:07.071545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:07.071895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] PATH_INFO: `/v3/users/97adac4405544fdabfceb9a56a66e36f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:07.073209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97adac4405544fdabfceb9a56a66e36f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:07.081665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:07.314261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97adac4405544fdabfceb9a56a66e36f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:07.323098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97adac4405544fdabfceb9a56a66e36f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:07.324284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97adac4405544fdabfceb9a56a66e36f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:07.325990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97adac4405544fdabfceb9a56a66e36f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:07.346158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97adac4405544fdabfceb9a56a66e36f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:07.479277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 995/1959] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:06 2022] DELETE /identity/v3/users/97adac4405544fdabfceb9a56a66e36f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:07.483825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:07.484678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:07.485318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:07.508607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:07.513280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:07.584013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 965/1960] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:07.593364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:07.594922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:07.665501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738a0a3-12af-48ce-91a2-6e90eb2210d6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:07.670439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5c017-96f5-444f-87d3-e1bbe9fd0401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:07.671173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5c017-96f5-444f-87d3-e1bbe9fd0401 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:07.671555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5c017-96f5-444f-87d3-e1bbe9fd0401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:07.672052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5c017-96f5-444f-87d3-e1bbe9fd0401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:07.682747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5c017-96f5-444f-87d3-e1bbe9fd0401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:07.683637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb5c017-96f5-444f-87d3-e1bbe9fd0401 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:07.695088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 996/1961] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:07.723303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:08.040453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:08.160393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:08.162357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:08.176728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d55246b-196b-4c8e-9b14-8feec9724626 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:08.181849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:08.182570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:08.182977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:08.183398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] PATH_INFO: `/v3/projects/28dbfbf36541410baae15996643dd687` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:08.184790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28dbfbf36541410baae15996643dd687)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:08.186964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:08.327452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28dbfbf36541410baae15996643dd687', 'request_id': 'req-0f3a0501-40e7-4f71-9a54-875896efe8a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:08.334359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28dbfbf36541410baae15996643dd687', 'request_id': 'req-0f3a0501-40e7-4f71-9a54-875896efe8a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:08.340298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28dbfbf36541410baae15996643dd687', 'request_id': 'req-0f3a0501-40e7-4f71-9a54-875896efe8a8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:08.433705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] The token cache is being invalidate because project 28dbfbf36541410baae15996643dd687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:08.434038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:08.434707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f3a0501-40e7-4f71-9a54-875896efe8a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:08.436389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 966/1962] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:08 2022] DELETE /identity/v3/projects/28dbfbf36541410baae15996643dd687 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:08.441276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:08.441772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:08.441863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:08.464182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:08.465700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:08.534246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 997/1963] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:08.542083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:08.542368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:08.542645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:08.565220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:08.566382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:08.576211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 967/1964] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:08.586710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:08.597523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11a633-fd34-4914-b628-3af2295432a8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:08.604216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:08.604918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:08.605271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:08.605610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:08.606404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:08.606807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:08.609016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:08.622844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 998/1965] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:08.629341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:08.640057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ed97f-4b9e-42af-94da-6e5dce99b3ac None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:08.644916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:08.645634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:08.645977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:08.646256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:08.651749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:08.651749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:08.777357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 968/1966] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:08 2022] POST /identity/v3/projects => generated 377 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:08.786830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:08.797759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1274-116f-437b-b799-1e6e31a47f59 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:08.880890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:08.881846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:08.882271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:08.882682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:08.883666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:08.886338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:09.001426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 999/1967] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:08 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:09.005780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:09.017186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6438a498-5938-47fb-a4b5-8d652d657ab8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:09.021878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:09.022496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:09.022778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:09.023054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:09.023700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:09.025339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:09.035965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 969/1968] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:09.040198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:09.050598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499447e0-2afc-46a9-9057-d3e4e8cf5737 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:09.055295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623bfcfa-b2e4-4425-8013-a4100918ef82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:09.056008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bfcfa-b2e4-4425-8013-a4100918ef82 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:09.056289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bfcfa-b2e4-4425-8013-a4100918ef82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:09.056563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623bfcfa-b2e4-4425-8013-a4100918ef82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:09.057206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bfcfa-b2e4-4425-8013-a4100918ef82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:09.058838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623bfcfa-b2e4-4425-8013-a4100918ef82 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:09.070657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1000/1969] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:09.074812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:09.086568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c0ae27-b0c3-4520-ae4c-ed71c43a60ce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:09.092017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:09.092818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:09.093113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:09.093392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] PATH_INFO: `/v3/projects/f516b72f92584a69bae8758e06e7b6d4/users/407f3f23c5444b2e824822343d3493ff/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:09.109498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f516b72f92584a69bae8758e06e7b6d4, user_id=407f3f23c5444b2e824822343d3493ff, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:09.113399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:09.220295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 970/1970] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:09 2022] PUT /identity/v3/projects/f516b72f92584a69bae8758e06e7b6d4/users/407f3f23c5444b2e824822343d3493ff/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:09.220404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:09.220404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:09.220404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:09.252667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] MFA Rules not processed for user `407f3f23c5444b2e824822343d3493ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:09.254234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:09.321754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1001/1971] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:09.335556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:09.337272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:09.360355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:09.406510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41e0fb-ef22-4bb7-9c7a-54ff4a624293 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:09.411964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492be109-6208-421d-8a8a-207c826dac39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:09.412514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492be109-6208-421d-8a8a-207c826dac39 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:09.412800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492be109-6208-421d-8a8a-207c826dac39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:09.413080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492be109-6208-421d-8a8a-207c826dac39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:09.421925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492be109-6208-421d-8a8a-207c826dac39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:09.423632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492be109-6208-421d-8a8a-207c826dac39 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:09.432603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 971/1972] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:09 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:09.560580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:09.562341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:09.575063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:09.635062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cb3079-578c-4e2f-a0dd-9a0fcdd0bc26 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:09.640849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:09.641556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:09.641873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:09.642157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] PATH_INFO: `/v3/users/0be006c783a2402bae8b8383d0f7bf33` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:09.643307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0be006c783a2402bae8b8383d0f7bf33)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:09.645251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:09.868141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0be006c783a2402bae8b8383d0f7bf33'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:09.877804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0be006c783a2402bae8b8383d0f7bf33'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:09.878797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0be006c783a2402bae8b8383d0f7bf33'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:09.879519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0be006c783a2402bae8b8383d0f7bf33'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:09.897061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0be006c783a2402bae8b8383d0f7bf33'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:10.012380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1002/1973] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:09 2022] DELETE /identity/v3/users/0be006c783a2402bae8b8383d0f7bf33 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:10.017633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:10.017989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:10.018272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:10.039165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:10.040392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:10.113060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 972/1974] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:10.120930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:10.123242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:10.199801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97b7f8-c88c-4f80-8619-08a6b5a24dc4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:10.201217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:10.201333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:10.201646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:10.201953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:10.212001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:10.213460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:10.228714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1003/1975] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:10.684519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:10.686840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:10.699303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb6768-7bf0-425d-8ef7-034aa086d587 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:10.704408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:10.705064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:10.705352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:10.705744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] PATH_INFO: `/v3/projects/8095bc61e93444e0b1912c431433a242` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:10.706938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8095bc61e93444e0b1912c431433a242)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:10.708878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:10.830867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8095bc61e93444e0b1912c431433a242', 'request_id': 'req-a7b32acf-5a97-4470-943c-c7c14203dff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:10.837761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8095bc61e93444e0b1912c431433a242', 'request_id': 'req-a7b32acf-5a97-4470-943c-c7c14203dff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:10.843315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8095bc61e93444e0b1912c431433a242', 'request_id': 'req-a7b32acf-5a97-4470-943c-c7c14203dff7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:10.888703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:10.920012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] The token cache is being invalidate because project 8095bc61e93444e0b1912c431433a242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:10.920352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:10.921011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:10.922848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 973/1976] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:10 2022] DELETE /identity/v3/projects/8095bc61e93444e0b1912c431433a242 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:10.926887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:10.937205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7b3962-4058-41d5-9c39-bf34212f0c3e service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:11.003170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:11.003822 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:11.004111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:11.004390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] PATH_INFO: `/v3/users/f2f9ca50c8f44e08a0d8bc69c58a6a41` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:11.005633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2f9ca50c8f44e08a0d8bc69c58a6a41)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:11.007893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:11.404517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2f9ca50c8f44e08a0d8bc69c58a6a41'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:11.415484 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2f9ca50c8f44e08a0d8bc69c58a6a41'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:11.416283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2f9ca50c8f44e08a0d8bc69c58a6a41'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:11.417051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2f9ca50c8f44e08a0d8bc69c58a6a41'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:11.430392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba9dcd6c-f60b-4c15-bfd5-23479d182ee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2f9ca50c8f44e08a0d8bc69c58a6a41'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:11.580132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1004/1977] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:10 2022] DELETE /identity/v3/users/f2f9ca50c8f44e08a0d8bc69c58a6a41 => generated 0 bytes in 654 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:11.961123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:12.108751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:12.110430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:12.129561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b32acf-5a97-4470-943c-c7c14203dff7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:12.187012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:12.205084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:12.205769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.206057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.206999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] PATH_INFO: `/v3/projects/7216364fa6b444478d94f860a518a121` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.208188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7216364fa6b444478d94f860a518a121)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:12.210183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:12.338029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7216364fa6b444478d94f860a518a121', 'request_id': 'req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:12.345220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7216364fa6b444478d94f860a518a121', 'request_id': 'req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:12.351159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7216364fa6b444478d94f860a518a121', 'request_id': 'req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:12.446299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] The token cache is being invalidate because project 7216364fa6b444478d94f860a518a121 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:12.446685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:12.447446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0c5a7089-eab9-4f04-9d5b-1610a059ddc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:12.449939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 974/1978] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:12 2022] DELETE /identity/v3/projects/7216364fa6b444478d94f860a518a121 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:12.456195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.456826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.457301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.494664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:12.496011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:12.566071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1005/1979] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:12.573775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.574079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.574382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.596423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:12.597852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:12.606152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 975/1980] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:12.611192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:12.621205 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f2bd2-63b5-45f1-bd80-c795ef77ea80 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:12.626159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:12.626776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.627059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.627338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.628013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:12.628502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:12.630169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:12.642490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1006/1981] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:12:12.647901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:12.660855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e800ebd-a879-4016-8e96-7bf2d888887a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:12.667431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:12.668148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.668530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.668897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.670601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:12.674345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:12.676159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:12.771131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 976/1982] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:12 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:12.782679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:12.787655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03199cd7-cfb1-41b7-8672-8d534e25c211 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:12.860515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:12.861284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.861639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.862021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.862729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:12.864942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:12.958006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1007/1983] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:12 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:12.962136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:12.973574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a97dc-a04b-4999-97a8-0f2d64c6adbc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:12.978369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:12.978990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:12.979289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:12.979532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:12.980209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:12.981942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:12.993141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 977/1984] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:12.997399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.008020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525649-f5f3-4005-855a-1ef28dd14721 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.012817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605c4d1-bb83-4eee-8853-1ab1f6cbde1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.014281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605c4d1-bb83-4eee-8853-1ab1f6cbde1d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.014588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605c4d1-bb83-4eee-8853-1ab1f6cbde1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.014893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605c4d1-bb83-4eee-8853-1ab1f6cbde1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.015558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605c4d1-bb83-4eee-8853-1ab1f6cbde1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.017387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605c4d1-bb83-4eee-8853-1ab1f6cbde1d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.029481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1008/1985] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:13.033858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.044427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de8f4db-23bb-4a55-85b1-f1fad5438462 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.056339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.057272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.057712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.058104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] PATH_INFO: `/v3/projects/3de52e6f98bc49adb792785e4ef412ec/users/c30d61e09b1f4b339e4dbaad14806ba8/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.077369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3de52e6f98bc49adb792785e4ef412ec, user_id=c30d61e09b1f4b339e4dbaad14806ba8, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.079980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.174975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=75,keystone:DELETE=22,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:13.181657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 978/1986] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:13 2022] PUT /identity/v3/projects/3de52e6f98bc49adb792785e4ef412ec/users/c30d61e09b1f4b339e4dbaad14806ba8/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:13.195831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.196406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.196689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.229714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] MFA Rules not processed for user `c30d61e09b1f4b339e4dbaad14806ba8`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:13.231224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:13.304127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1009/1987] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:13.308752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.401704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1edbd1-1807-4bbd-b6c6-fdfa70d121b8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.407846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.409157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.409459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.409762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.412431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.416004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.575426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 979/1988] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:13 2022] POST /identity/v3/projects => generated 385 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:13.580366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.591608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593997c-fdb6-4d6f-9096-034473ca2126 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.655057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.655774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.656125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.656469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.657606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.659425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.674563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:13.737764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1010/1989] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:13 2022] POST /identity/v3/users => generated 299 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:13.741918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.756884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef205bb-325c-4e70-b4bd-e6f550dfdec5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.765827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.766734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.767248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.769187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.770108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.771760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.784989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 980/1990] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:13.789248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.805118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54c354f-06c8-4924-b316-a45de818aef0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.811561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.812259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.812545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.812826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] PATH_INFO: `/v3/projects/41453067593d4b2c964a23c09916739f/users/e7ffccb19b124abb928d2298e37d56f4/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.839139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] RBAC: Authorizing `identity:create_grant(project_id=41453067593d4b2c964a23c09916739f, user_id=e7ffccb19b124abb928d2298e37d56f4, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.842586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:13.914913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1011/1991] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:13 2022] PUT /identity/v3/projects/41453067593d4b2c964a23c09916739f/users/e7ffccb19b124abb928d2298e37d56f4/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:13.920098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:13.930931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3a6e9-65f2-4550-9853-1bfd254348d1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:13.938298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:13.990693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ba36a0-0376-4ba3-9389-43622760da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:13.991423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba36a0-0376-4ba3-9389-43622760da07 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:13.991580 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba36a0-0376-4ba3-9389-43622760da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:13.991862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ba36a0-0376-4ba3-9389-43622760da07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:13.992502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba36a0-0376-4ba3-9389-43622760da07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:13.994295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ba36a0-0376-4ba3-9389-43622760da07 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:14.005050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 981/1992] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:14.009255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:14.019753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39476c-9664-42d1-b17c-0649f003e22e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:14.024527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:14.025183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:14.025465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:14.025765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] PATH_INFO: `/v3/projects/41453067593d4b2c964a23c09916739f/users/e7ffccb19b124abb928d2298e37d56f4/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:14.027687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41453067593d4b2c964a23c09916739f, user_id=e7ffccb19b124abb928d2298e37d56f4, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:14.031053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:14.041595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1012/1993] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:14 2022] PUT /identity/v3/projects/41453067593d4b2c964a23c09916739f/users/e7ffccb19b124abb928d2298e37d56f4/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:14.054992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fb8bc-0f0a-4775-9063-c0e267c0e69d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:14.055431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fb8bc-0f0a-4775-9063-c0e267c0e69d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:14.055940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5fb8bc-0f0a-4775-9063-c0e267c0e69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:14.087355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2a5fb8bc-0f0a-4775-9063-c0e267c0e69d None None] MFA Rules not processed for user `e7ffccb19b124abb928d2298e37d56f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:14.088644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2a5fb8bc-0f0a-4775-9063-c0e267c0e69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:14.163697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:14.176253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 982/1994] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:14.187909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:14.193038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:14.286880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9a33e3-deac-450c-a695-552a4a8f9f47 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:14.290640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af692f20-17fe-4cb3-98f9-05654c931487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:14.291230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af692f20-17fe-4cb3-98f9-05654c931487 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:14.291515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af692f20-17fe-4cb3-98f9-05654c931487 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:14.291789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af692f20-17fe-4cb3-98f9-05654c931487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:14.351442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af692f20-17fe-4cb3-98f9-05654c931487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:14.352821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af692f20-17fe-4cb3-98f9-05654c931487 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:14.362606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1013/1995] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:14.506411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:14.552214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:15.049919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:15.174936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:15.283433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:15.901355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255e63b-d9c3-4578-9ca9-6757be868639 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:15.901650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255e63b-d9c3-4578-9ca9-6757be868639 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:15.901962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255e63b-d9c3-4578-9ca9-6757be868639 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:15.903324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 983/1996] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:12:15.932342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8e4b9-c51d-41d6-915e-673ac13a022b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:15.932764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8e4b9-c51d-41d6-915e-673ac13a022b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:15.933091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8e4b9-c51d-41d6-915e-673ac13a022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:15.964183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9cc8e4b9-c51d-41d6-915e-673ac13a022b None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:15.966515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9cc8e4b9-c51d-41d6-915e-673ac13a022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:16.037874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1014/1997] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:12:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:16.050528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a1d32-fde2-4809-8ac4-f3e5560a3efe None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:16.050820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a1d32-fde2-4809-8ac4-f3e5560a3efe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:16.051099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a1d32-fde2-4809-8ac4-f3e5560a3efe None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:16.052428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 984/1998] 10.222.0.52 () {54 vars in 961 bytes} [Tue Aug 30 01:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:12:16.057947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:16.058506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:16.058677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:16.079169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:16.080376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:16.084528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:16.092704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1015/1999] 10.222.0.52 () {58 vars in 1058 bytes} [Tue Aug 30 01:12:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:16.098813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a1d32-fde2-4809-8ac4-f3e5560a3efe None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:16.100617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-de3a1d32-fde2-4809-8ac4-f3e5560a3efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:16.101062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:16.112932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66cebe-36a2-421c-ab50-db98d70fc75d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:16.118421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:16.119284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:16.119399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:16.119654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:16.129303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:16.130935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:16.140769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1016/2000] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:16.182379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a1d32-fde2-4809-8ac4-f3e5560a3efe None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:16.188607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:16.189515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:16.189946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:16.190124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] PATH_INFO: `/v3/users/13848c47ddd9441282bec982cccc8b91` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:16.192273 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13848c47ddd9441282bec982cccc8b91)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:16.193627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:16.254416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:16.583031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13848c47ddd9441282bec982cccc8b91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:16.590376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13848c47ddd9441282bec982cccc8b91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:16.711769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13848c47ddd9441282bec982cccc8b91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:16.712453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13848c47ddd9441282bec982cccc8b91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:16.727052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-15dcab61-8e7f-4330-aa1f-335bb44c0c91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13848c47ddd9441282bec982cccc8b91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:16.733008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 985/2001] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:16 2022] DELETE /identity/v3/users/13848c47ddd9441282bec982cccc8b91 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:16.821912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:17.010740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.022768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a010507b-aa70-4191-a43c-a994fc399c56 service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.072074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:17.110503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.111169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.111453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.116126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] PATH_INFO: `/v3/projects/4f082e13ad014501b7a781167ddb2ed8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.116126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f082e13ad014501b7a781167ddb2ed8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.116126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.222551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f082e13ad014501b7a781167ddb2ed8', 'request_id': 'req-96484812-c7e3-4555-916e-5e5d45cf8848'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:17.351392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f082e13ad014501b7a781167ddb2ed8', 'request_id': 'req-96484812-c7e3-4555-916e-5e5d45cf8848'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:17.358807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f082e13ad014501b7a781167ddb2ed8', 'request_id': 'req-96484812-c7e3-4555-916e-5e5d45cf8848'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:17.369238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] The token cache is being invalidate because project 4f082e13ad014501b7a781167ddb2ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:17.369582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:17.370287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-96484812-c7e3-4555-916e-5e5d45cf8848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:17.371993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1017/2002] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:17 2022] DELETE /identity/v3/projects/4f082e13ad014501b7a781167ddb2ed8 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:17.377661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.378186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.378276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.399339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:17.400572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:17.476028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 986/2003] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:17.485656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.485997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.486247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.506927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:17.508114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:17.516435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1018/2004] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:17.520799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.530713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8fc09f-e183-47c1-94cc-f68a64359a36 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.535428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.536091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.536407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.536664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.537314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:17.537626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.537928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:17.539310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.556768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 987/2005] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:12:17.558616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.568951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35e29d-8876-424d-a42b-7f67da44ddb5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.574139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.574758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.575038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.575356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.576731 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.578739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.693838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1019/2006] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:17 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:17.698252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.710165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca22ac-e218-4de2-b84f-423bd66a6196 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.772218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.773221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.773718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.774246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.775393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.779597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.789619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:17.894769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 988/2007] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:17 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:17.898607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.913291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76729-a34c-4282-8256-520c093ca048 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.918383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.919009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.920139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.920478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.921236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.923167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.936510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1020/2008] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:17.941504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.953150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f71af10-0008-48e8-92fe-fea7093624f0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.959442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153c315-2412-497b-ac09-9e2bcf7d0729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.960145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153c315-2412-497b-ac09-9e2bcf7d0729 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.960428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153c315-2412-497b-ac09-9e2bcf7d0729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.960773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153c315-2412-497b-ac09-9e2bcf7d0729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:17.961476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153c315-2412-497b-ac09-9e2bcf7d0729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:17.963178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153c315-2412-497b-ac09-9e2bcf7d0729 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:17.976294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 989/2009] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:17.980604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:17.991568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a396494c-ab5e-4cd8-b44f-66029ccf3960 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:17.996445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:17.997110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:17.997499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:17.997744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] PATH_INFO: `/v3/projects/750b277bc8ea405a89e6279c5e2c529d/users/a80403bf66a84535965bc40a46e9abfc/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:18.013440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=750b277bc8ea405a89e6279c5e2c529d, user_id=a80403bf66a84535965bc40a46e9abfc, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:18.016144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:18.039993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:18.104761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1021/2010] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:17 2022] PUT /identity/v3/projects/750b277bc8ea405a89e6279c5e2c529d/users/a80403bf66a84535965bc40a46e9abfc/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:18.116691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:18.117015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:18.117295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:18.147827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] MFA Rules not processed for user `a80403bf66a84535965bc40a46e9abfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:18.149023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:18.244748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 990/2011] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:18.256814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:18.259848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:18.290077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:18.312739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:18.331161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d433f5d-0dee-4549-99c5-8be83bb5c33c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:18.335493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96314491-1e4f-4a9b-98cd-75f31f77d9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:18.336120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96314491-1e4f-4a9b-98cd-75f31f77d9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:18.336408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96314491-1e4f-4a9b-98cd-75f31f77d9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:18.336689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96314491-1e4f-4a9b-98cd-75f31f77d9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:18.346331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96314491-1e4f-4a9b-98cd-75f31f77d9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:18.347786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96314491-1e4f-4a9b-98cd-75f31f77d9b9 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:18.357155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1022/2012] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:18.480656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:18.571828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:18.883465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:19.332215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:19.671754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:24.011400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:24.048015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa616245-45fe-457e-b2e2-7768ca29697c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:24.055631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:24.056843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:24.057689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:24.058194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:24.061526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:24.140920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:24.142181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:24.152745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 991/2013] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:27.427311 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f74701-df26-4c68-ad40-40c8152faf43 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:27.427884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f74701-df26-4c68-ad40-40c8152faf43 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:27.427967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f74701-df26-4c68-ad40-40c8152faf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:27.448719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-50f74701-df26-4c68-ad40-40c8152faf43 None None] MFA Rules not processed for user `1b225b55d18740c2833cade52edc556e`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:27.449868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-50f74701-df26-4c68-ad40-40c8152faf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:27.498772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:27.517233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1023/2014] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:33.364461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:33.451426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbec063-0da0-4b80-b940-986285ced383 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:33.458509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:33.459523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.459810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.460087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] PATH_INFO: `/v3/users/a80403bf66a84535965bc40a46e9abfc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.462518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80403bf66a84535965bc40a46e9abfc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:33.466475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:33.803518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:33.809160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80403bf66a84535965bc40a46e9abfc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:33.820077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80403bf66a84535965bc40a46e9abfc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:33.928119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80403bf66a84535965bc40a46e9abfc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:33.928898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80403bf66a84535965bc40a46e9abfc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:33.959275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80403bf66a84535965bc40a46e9abfc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:33.964710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 992/2015] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:33 2022] DELETE /identity/v3/users/a80403bf66a84535965bc40a46e9abfc => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:33.969771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:33.970263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:33.970382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:33.992771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:33.998696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:34.065038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1024/2016] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:34.075839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:34.077329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:34.141246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a98056-0c1f-428b-a569-aec2819e0d7e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:34.144922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:34.145531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:34.145840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:34.146119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:34.155718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:34.157830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:34.167963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 993/2017] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:34.640895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:34.642398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:34.655178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410fee83-06e5-4469-a66f-8a07b96a2a5f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:34.661851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:34.662384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:34.662662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:34.662933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] PATH_INFO: `/v3/projects/750b277bc8ea405a89e6279c5e2c529d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:34.664140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=750b277bc8ea405a89e6279c5e2c529d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:34.666176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:34.852717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '750b277bc8ea405a89e6279c5e2c529d', 'request_id': 'req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.058231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.059817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '750b277bc8ea405a89e6279c5e2c529d', 'request_id': 'req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.067549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '750b277bc8ea405a89e6279c5e2c529d', 'request_id': 'req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.072089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.080716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] The token cache is being invalidate because project 750b277bc8ea405a89e6279c5e2c529d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:35.081013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.081681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a800a8fd-d6eb-4c9b-b9f2-26405b577ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.082539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac295a40-ff13-4a10-b0c5-5ac9635c044b service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.083234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1025/2018] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:34 2022] DELETE /identity/v3/projects/750b277bc8ea405a89e6279c5e2c529d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:35.087831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738108d8-9155-4a4a-b459-89a877aa11fe None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.088118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738108d8-9155-4a4a-b459-89a877aa11fe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.088402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738108d8-9155-4a4a-b459-89a877aa11fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.120250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-738108d8-9155-4a4a-b459-89a877aa11fe None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:35.121468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-738108d8-9155-4a4a-b459-89a877aa11fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.149906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.150562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.150851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.151247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] PATH_INFO: `/v3/users/c30d61e09b1f4b339e4dbaad14806ba8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.152336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c30d61e09b1f4b339e4dbaad14806ba8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.154515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.189747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1026/2019] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:35.199149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.201764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.201764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.220007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:35.221222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.229407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1027/2020] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:35.233514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.242801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05f2c8-aa51-48b5-93e5-86b3951179a9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.247455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.248039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.248353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.248628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.249276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:35.249595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.252021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.254629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:35.263891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1028/2021] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:12:35.269107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.279035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37abc1bf-d1cb-462a-8dac-155e6b472d30 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.284022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.284631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.284906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.285185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.285844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.287754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.407844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c30d61e09b1f4b339e4dbaad14806ba8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.415262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c30d61e09b1f4b339e4dbaad14806ba8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.445956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1029/2022] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:35.449069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.512388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127fe66d-1fbd-46cb-8692-9f5bece9bbe3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.517197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.517849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.518117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.518388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.519017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.521143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.522839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c30d61e09b1f4b339e4dbaad14806ba8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.523511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c30d61e09b1f4b339e4dbaad14806ba8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.525844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:35.537162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55da8668-6800-4793-96a7-d05d4215cc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c30d61e09b1f4b339e4dbaad14806ba8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:35.543257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 994/2023] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:35 2022] DELETE /identity/v3/users/c30d61e09b1f4b339e4dbaad14806ba8 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:35.548371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.548666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.548949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.568616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:35.569863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.641102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 995/2024] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:35.647057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1030/2025] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/users => generated 286 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:35.648823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.651632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.655464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.660763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.667924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4901c6-f88a-49b9-bec9-29c7b3e9cf6b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.673908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.674725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.675097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.675540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.676241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.677940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.692104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 996/2026] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:35.693237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.704880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c76191e-33af-44c9-a34e-092f13313b47 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.710478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.711129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.711462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.711744 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.712388 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.714202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.719150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db806d2a-ef94-4698-a2b3-af7fab9efa8f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.728650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 997/2027] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:35.729878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:35.740699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57607f-4463-4431-bb51-0c8b8f1e1e08 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:35.746337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.750438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.750438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.750438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] PATH_INFO: `/v3/projects/bec70e149d0542bc8c1bcc9f29083f46/users/a167b5065cc54f0186af5596408b234c/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.764510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bec70e149d0542bc8c1bcc9f29083f46, user_id=a167b5065cc54f0186af5596408b234c, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.768460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.772133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d2de6-a77f-4823-8aae-d84b0c20fba1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:35.772823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d2de6-a77f-4823-8aae-d84b0c20fba1 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.773111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d2de6-a77f-4823-8aae-d84b0c20fba1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.773386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d2de6-a77f-4823-8aae-d84b0c20fba1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.782880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d2de6-a77f-4823-8aae-d84b0c20fba1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:35.784314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d2de6-a77f-4823-8aae-d84b0c20fba1 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:35.793803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1031/2028] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:35.832122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 998/2029] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:35 2022] PUT /identity/v3/projects/bec70e149d0542bc8c1bcc9f29083f46/users/a167b5065cc54f0186af5596408b234c/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:35.843053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:35.843647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:35.843903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:35.874188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] MFA Rules not processed for user `a167b5065cc54f0186af5596408b234c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:35.875971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:35.886022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:35.941931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1032/2030] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:35.946553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.038554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7948bc-119a-4a4c-96c9-dbd135044931 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.045349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.047635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.047635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.047840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.048790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.050854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.175212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 999/2031] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:35 2022] POST /identity/v3/projects => generated 371 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:36.179480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.193361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d6752f-f73d-4a3b-bc15-208388a9ea27 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.260611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.261395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.261686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.261999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.262636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.266378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.267571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.270690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:36.285339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22b17d-e2a9-46f6-89e6-48ab9aeb0deb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.315789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.317198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.317484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.318146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] PATH_INFO: `/v3/projects/3de52e6f98bc49adb792785e4ef412ec` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.319364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3de52e6f98bc49adb792785e4ef412ec)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.321300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.348692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:36.393320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1033/2032] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:36 2022] POST /identity/v3/users => generated 292 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:36.399952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.418074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c713b3e8-168d-48a6-a2e0-1f67ab60ed53 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.425713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.428560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.428560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.428560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.429167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.432217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.450782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1034/2033] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:36.460665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.492099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16ec83-a1a2-4522-acba-445b65ea3752 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.497379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.498086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.498375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.499061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] PATH_INFO: `/v3/projects/2db63e2c839c4583ad050d2243656a45/users/589f69ef16d545f8bda4559bbbeeae6c/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.516616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db63e2c839c4583ad050d2243656a45, user_id=589f69ef16d545f8bda4559bbbeeae6c, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.519695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.538406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3de52e6f98bc49adb792785e4ef412ec', 'request_id': 'req-f3c1ecc1-1579-4244-acee-428accf2d896'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:36.545265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3de52e6f98bc49adb792785e4ef412ec', 'request_id': 'req-f3c1ecc1-1579-4244-acee-428accf2d896'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:36.556768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3de52e6f98bc49adb792785e4ef412ec', 'request_id': 'req-f3c1ecc1-1579-4244-acee-428accf2d896'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:36.690073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1035/2034] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:36 2022] PUT /identity/v3/projects/2db63e2c839c4583ad050d2243656a45/users/589f69ef16d545f8bda4559bbbeeae6c/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:36.694064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.725926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] The token cache is being invalidate because project 3de52e6f98bc49adb792785e4ef412ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:36.726147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:36.726802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:36.729084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1000/2035] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:36 2022] DELETE /identity/v3/projects/3de52e6f98bc49adb792785e4ef412ec => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:36.733607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.735067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:36.792451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c92c3-bcc9-4214-873a-218d9aec1783 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.798519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.799157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.799428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.799691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.800346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.801989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.813216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1036/2036] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:36.817175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1ecc1-1579-4244-acee-428accf2d896 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.818140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.819809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:36.824212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.824877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.825237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.826190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] PATH_INFO: `/v3/users/e7ffccb19b124abb928d2298e37d56f4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.827366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7ffccb19b124abb928d2298e37d56f4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.828035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:36.829839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.833807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8c96da-083f-4c38-88c5-199aedadbc42 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:36.840166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc3901-4df8-4d16-a9fc-e021aed9743c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:36.840995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc3901-4df8-4d16-a9fc-e021aed9743c admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.841285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc3901-4df8-4d16-a9fc-e021aed9743c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.841564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc3901-4df8-4d16-a9fc-e021aed9743c admin admin] PATH_INFO: `/v3/projects/2db63e2c839c4583ad050d2243656a45/users/589f69ef16d545f8bda4559bbbeeae6c/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.844086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc3901-4df8-4d16-a9fc-e021aed9743c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2db63e2c839c4583ad050d2243656a45, user_id=589f69ef16d545f8bda4559bbbeeae6c, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:36.847060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc3901-4df8-4d16-a9fc-e021aed9743c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:36.862637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1037/2037] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:36 2022] PUT /identity/v3/projects/2db63e2c839c4583ad050d2243656a45/users/589f69ef16d545f8bda4559bbbeeae6c/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:36.873146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:36.873540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:36.873998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:36.903303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] MFA Rules not processed for user `589f69ef16d545f8bda4559bbbeeae6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:36.909392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:36.975278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1038/2038] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:36 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:12:36.991375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:36.994041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:37.057634 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3946067f-84a6-41bf-8ec6-e2aaca61477a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:37.061691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:37.063099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:37.063381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:37.063656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:37.073772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:37.075689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:37.079559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7ffccb19b124abb928d2298e37d56f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:37.085220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1039/2039] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:36 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:37.087245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7ffccb19b124abb928d2298e37d56f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:37.226959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7ffccb19b124abb928d2298e37d56f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:37.229045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7ffccb19b124abb928d2298e37d56f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:37.255600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7ffccb19b124abb928d2298e37d56f4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:37.263044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1001/2040] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:36 2022] DELETE /identity/v3/users/e7ffccb19b124abb928d2298e37d56f4 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:37.418551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:37.427792 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf1c8a-a35e-4f05-9705-7a2c45ce05cb service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:37.484495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:37.484495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:37.484495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:37.484861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:37.486647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:37.521635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:37.579613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:37.581016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836b9bd-6c23-4ee1-8597-12644be78a8d service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:37.591548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1040/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:37.880022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:37.884331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:37.937790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:37.979661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db0a546-1acf-4281-9085-aa5e21e0521e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:37.985025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:37.985797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:37.986083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:37.986359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] PATH_INFO: `/v3/projects/41453067593d4b2c964a23c09916739f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:37.987484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41453067593d4b2c964a23c09916739f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:37.989389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:38.173841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41453067593d4b2c964a23c09916739f', 'request_id': 'req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:38.184444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41453067593d4b2c964a23c09916739f', 'request_id': 'req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:38.193973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41453067593d4b2c964a23c09916739f', 'request_id': 'req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:38.313996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] The token cache is being invalidate because project 41453067593d4b2c964a23c09916739f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:38.314564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:38.318135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-aab129d5-8edd-4d43-a9fa-c34a68d1fce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:38.320279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1002/2042] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:37 2022] DELETE /identity/v3/projects/41453067593d4b2c964a23c09916739f => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:38.326652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:38.331521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:38.331521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:38.354862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:38.356530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:38.432920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1041/2043] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:38.442215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:38.442501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:38.442784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:38.473642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:38.475176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:38.485518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1003/2044] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:38.491277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:38.510587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde85f98-d4b5-4ab3-b835-b27b074871bd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:38.517031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:38.518772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:38.518772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:38.519044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:38.520207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:38.520734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:38.523438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:38.542456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1042/2045] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:12:38.547843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:38.557126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9b0323-92cd-4cbe-b56f-540ac67daa09 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:38.562099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:38.562822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:38.563352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:38.563700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:38.564351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:38.566391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:38.711925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1004/2046] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:38 2022] POST /identity/v3/projects => generated 399 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:12:38.716850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:38.732422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad63a9-1fbf-4131-b35c-e1db0981ebbf admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:38.819032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:38.819995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:38.820346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:38.831133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:38.841755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:38.841755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:39.016195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1043/2047] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:38 2022] POST /identity/v3/users => generated 300 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:39.020633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:39.031954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08792e2-e187-4a3e-92a7-fa9727d808c6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:39.036923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:39.037470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:39.037939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:39.038222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:39.038875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:39.040762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:39.057044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1005/2048] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:39.058358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:39.072050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bbd612-5c37-46a5-ac43-1a1e16a8760e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:39.078145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68149fba-e857-4610-866a-864d3b688d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:39.079908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68149fba-e857-4610-866a-864d3b688d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:39.080594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68149fba-e857-4610-866a-864d3b688d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:39.080847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68149fba-e857-4610-866a-864d3b688d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:39.083295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68149fba-e857-4610-866a-864d3b688d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:39.088789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68149fba-e857-4610-866a-864d3b688d09 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:39.100887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1044/2049] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:12:39.105166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:39.116351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b28796-0bd3-4b80-a283-7cdb4a242f19 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:39.121383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:39.122093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:39.122468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:39.122725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] PATH_INFO: `/v3/projects/beeecdb46ebc4ffc8e8230be3f0c5dae/users/353e182ee28440d18416864adbcd0b3b/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:39.151188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beeecdb46ebc4ffc8e8230be3f0c5dae, user_id=353e182ee28440d18416864adbcd0b3b, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:39.154022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:39.301281 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1006/2050] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:39 2022] PUT /identity/v3/projects/beeecdb46ebc4ffc8e8230be3f0c5dae/users/353e182ee28440d18416864adbcd0b3b/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:39.314690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:39.315252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:39.315636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:39.350291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] MFA Rules not processed for user `353e182ee28440d18416864adbcd0b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:39.352159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:39.382745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:39.427863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1045/2051] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:39.440306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:39.442277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:39.510685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cffe94f-fc4f-4881-9ef8-776370721951 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:39.515681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcdf803-cec8-4fb2-81f8-bb3e97019c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:39.521882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdf803-cec8-4fb2-81f8-bb3e97019c90 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:39.521882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdf803-cec8-4fb2-81f8-bb3e97019c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:39.521882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcdf803-cec8-4fb2-81f8-bb3e97019c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:39.523877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:39.527572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdf803-cec8-4fb2-81f8-bb3e97019c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:39.528992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcdf803-cec8-4fb2-81f8-bb3e97019c90 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:39.539519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1007/2052] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:39 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:40.386516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:40.388150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:40.457987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0ee24b-4068-4747-80c5-33e1f2725015 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:40.462646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:40.463262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:40.463540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:40.463813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] PATH_INFO: `/v3/users/407f3f23c5444b2e824822343d3493ff` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:40.464931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=407f3f23c5444b2e824822343d3493ff)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:40.466896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:40.821425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '407f3f23c5444b2e824822343d3493ff'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:40.830464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '407f3f23c5444b2e824822343d3493ff'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:40.831433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '407f3f23c5444b2e824822343d3493ff'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:40.832392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '407f3f23c5444b2e824822343d3493ff'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:40.847878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '407f3f23c5444b2e824822343d3493ff'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:40.994812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1046/2053] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:40 2022] DELETE /identity/v3/users/407f3f23c5444b2e824822343d3493ff => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:41.000470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:41.001414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:41.001795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:41.023698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:41.024910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:41.085361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1008/2054] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:41.095692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:41.097109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:41.162540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9603642-586e-40f7-a020-11f04669fd9b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:41.166653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8929ab59-a7b2-46ed-b634-4caefa8f831a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:41.167431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ab59-a7b2-46ed-b634-4caefa8f831a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:41.167700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ab59-a7b2-46ed-b634-4caefa8f831a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:41.167976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8929ab59-a7b2-46ed-b634-4caefa8f831a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:41.178214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ab59-a7b2-46ed-b634-4caefa8f831a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:41.179734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8929ab59-a7b2-46ed-b634-4caefa8f831a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:41.191119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1047/2055] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:41.686579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:41.688151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:41.702255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a145d5d4-1c0b-4a43-a983-330d86d7f9f5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:41.709183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:41.720707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:41.721484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:41.721884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] PATH_INFO: `/v3/projects/f516b72f92584a69bae8758e06e7b6d4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:41.724267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f516b72f92584a69bae8758e06e7b6d4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:41.726639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:41.868631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f516b72f92584a69bae8758e06e7b6d4', 'request_id': 'req-6732a562-4aca-4e39-8a38-801a8a0b3eff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:41.875175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f516b72f92584a69bae8758e06e7b6d4', 'request_id': 'req-6732a562-4aca-4e39-8a38-801a8a0b3eff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:41.880650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f516b72f92584a69bae8758e06e7b6d4', 'request_id': 'req-6732a562-4aca-4e39-8a38-801a8a0b3eff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:41.989997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] The token cache is being invalidate because project f516b72f92584a69bae8758e06e7b6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:41.990323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:41.991940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6732a562-4aca-4e39-8a38-801a8a0b3eff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:41.997500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1009/2056] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:41 2022] DELETE /identity/v3/projects/f516b72f92584a69bae8758e06e7b6d4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:41.999213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:41.999817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:41.999945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.012028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:42.025784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:42.027172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:42.105708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1048/2057] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:42.115564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.116521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.116963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.143775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:42.145212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:42.156972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1010/2058] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:42.160981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.171403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a9ed3-418b-49a7-b120-216b63d57823 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.176646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.177247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.177528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.177832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.178475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:12:42.178816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.180433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.192582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1049/2059] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:12:42.196985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.207175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7126b4-fb2d-4565-ace7-3a10c40bcef0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.211996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.212599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.212883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.213164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.213872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.215763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.340916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1011/2060] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:42.346042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.356543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a635b5bd-db88-4bac-8ab0-915d73595e03 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.416324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.416936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.417221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.417498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.418299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.420255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.548763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1050/2061] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:42.553072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.564553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c57a2e-1a8a-4ac1-8ff9-d5e440dff292 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.569422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.570073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.574523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.574523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.574523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.574523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.584837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1012/2062] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:42.588893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.600069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691532bc-2d6a-44b8-8dc3-4995f88c961b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.605068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbdbc6-598f-4a39-b38a-be44ea701989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.609018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbdbc6-598f-4a39-b38a-be44ea701989 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.609018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbdbc6-598f-4a39-b38a-be44ea701989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.609018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbdbc6-598f-4a39-b38a-be44ea701989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.609018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbdbc6-598f-4a39-b38a-be44ea701989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.609018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbdbc6-598f-4a39-b38a-be44ea701989 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.620120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1051/2063] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:42.624987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.635235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d51a5-8a82-42ab-96d1-30bcf82a6be7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.639991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.641259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.641536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.641856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] PATH_INFO: `/v3/projects/093b02e628a944929ab40fcf2bc07b25/users/e3227dfdb4c4453291f640e418ebebae/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.657054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=093b02e628a944929ab40fcf2bc07b25, user_id=e3227dfdb4c4453291f640e418ebebae, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.659772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:42.758101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1013/2064] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:42 2022] PUT /identity/v3/projects/093b02e628a944929ab40fcf2bc07b25/users/e3227dfdb4c4453291f640e418ebebae/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:42.770340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.770753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.771129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.800130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] MFA Rules not processed for user `e3227dfdb4c4453291f640e418ebebae`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:42.801349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:42.864811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1052/2065] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:42.869557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:42.932757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dd65c3-41f5-4ee5-972f-0bb7cd15b561 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:42.938368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:42.939035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:42.939317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:42.939588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:42.940232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:42.942203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.044229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1014/2066] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:42 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:43.049094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.060000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bde7bbf-76f6-441d-80fc-50bcfbf372d4 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.117528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.118171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.118431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.118708 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.119355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.121320 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.209472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1053/2067] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:43.210788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.222169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd77473-245d-442c-acb2-2f531da90c50 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.229458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.229458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.229458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.229458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.235735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.235735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.250489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1015/2068] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:43.251885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.262816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e139817-aa0d-499e-8a97-6e348ba8e3f3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.269511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4cc613-9706-46e7-b514-cd812f9c5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.271056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cc613-9706-46e7-b514-cd812f9c5898 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.271056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cc613-9706-46e7-b514-cd812f9c5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.271056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4cc613-9706-46e7-b514-cd812f9c5898 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.271865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cc613-9706-46e7-b514-cd812f9c5898 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.273503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4cc613-9706-46e7-b514-cd812f9c5898 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.287522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1054/2069] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:43.294724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.306366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32fb407-f6d5-47ac-82e9-a5805aaf27ca admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.313357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.314497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.314962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.315501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] PATH_INFO: `/v3/projects/ba8582f36aee4e579b408921af9061fa/users/62d85a3358ea4f26be9e8d57ab728f6c/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.328866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:43.338733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8582f36aee4e579b408921af9061fa, user_id=62d85a3358ea4f26be9e8d57ab728f6c, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.341654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.552070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1016/2070] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:43 2022] PUT /identity/v3/projects/ba8582f36aee4e579b408921af9061fa/users/62d85a3358ea4f26be9e8d57ab728f6c/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:43.564007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.572944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.572944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.595770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] MFA Rules not processed for user `62d85a3358ea4f26be9e8d57ab728f6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:43.597006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:43.665329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1055/2071] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:43.676559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.744850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca4f8-db9e-4b51-b95b-416b33c24866 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.750094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.750570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.750847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.751117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.751758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.753716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:43.872302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1017/2072] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:43.876447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:43.887093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6445546-36a1-4da4-9516-30b458bb6050 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:43.943783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:43.944654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:43.944938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:43.945217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:43.945992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:43.947831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.039361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1056/2073] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:12:43 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:12:44.043938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.055904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988f715c-5ed0-44b6-a260-21241895802e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.060635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.061246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.061528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.061826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.062983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.063439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.064572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.076619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1018/2074] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:44.081591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.091857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87001dcd-86ff-4e85-90a1-4a43fa431a50 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.096672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.097330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.097615 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.097927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] PATH_INFO: `/v3/projects/edbbf7fcf85644bfa4c241e61f3ace18/users/1a18dda92c12472f857c511be82526d8/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.112736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbbf7fcf85644bfa4c241e61f3ace18, user_id=1a18dda92c12472f857c511be82526d8, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.115429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.211092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1057/2075] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:44 2022] PUT /identity/v3/projects/edbbf7fcf85644bfa4c241e61f3ace18/users/1a18dda92c12472f857c511be82526d8/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:44.215563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.226122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d91a155-7c5c-48c7-87d3-e009bd2cb73a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.284249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa82348-c9c8-4476-9034-20df47cfba29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.284864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa82348-c9c8-4476-9034-20df47cfba29 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.285148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa82348-c9c8-4476-9034-20df47cfba29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.285427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa82348-c9c8-4476-9034-20df47cfba29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.286107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa82348-c9c8-4476-9034-20df47cfba29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.287792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa82348-c9c8-4476-9034-20df47cfba29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.299199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1019/2076] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:12:44.303303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.314590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f320813-b98a-4c0b-8287-af04d8828fbf admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.319303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.320838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.320838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.320838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] PATH_INFO: `/v3/projects/edbbf7fcf85644bfa4c241e61f3ace18/users/1a18dda92c12472f857c511be82526d8/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.322667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edbbf7fcf85644bfa4c241e61f3ace18, user_id=1a18dda92c12472f857c511be82526d8, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.325424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.337166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1058/2077] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:12:44 2022] PUT /identity/v3/projects/edbbf7fcf85644bfa4c241e61f3ace18/users/1a18dda92c12472f857c511be82526d8/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:44.347466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.347760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.348040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.378798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] MFA Rules not processed for user `1a18dda92c12472f857c511be82526d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:44.380023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.458379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1020/2078] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:44 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:44.470832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:44.472495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:44.550787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014a4683-bf92-4def-859b-fb27cccf6bd8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:44.555378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:44.556202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:44.556487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:44.556766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:44.599624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:44.616136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:44.617576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:44.626767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1059/2079] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:44.705806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:49.940734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:49.942435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:49.958322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75468f6-f420-4a9d-b8d2-b0018f063f15 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:49.962182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:49.962871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:49.963230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:49.963511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:50.027968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:50.027968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:50.035357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1021/2080] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:51.680350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:51.690166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366b18ee-fefb-416f-a5d2-0f5b898147b6 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:51.694104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967da098-8d84-47a1-bf35-f3eac56d037d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:51.694718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967da098-8d84-47a1-bf35-f3eac56d037d service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:51.694996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967da098-8d84-47a1-bf35-f3eac56d037d service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:51.695323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967da098-8d84-47a1-bf35-f3eac56d037d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:51.706273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967da098-8d84-47a1-bf35-f3eac56d037d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:51.707793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967da098-8d84-47a1-bf35-f3eac56d037d service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:51.719022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1060/2081] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:52.314306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:52.318066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:52.386928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:52.392028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51f838f-fcda-4576-b07c-60ac8c51ef50 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:52.396738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:52.397474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:52.397780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:52.398063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] PATH_INFO: `/v3/users/353e182ee28440d18416864adbcd0b3b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:52.402229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353e182ee28440d18416864adbcd0b3b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:52.402229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:52.621010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353e182ee28440d18416864adbcd0b3b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:52.633949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353e182ee28440d18416864adbcd0b3b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:52.730683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353e182ee28440d18416864adbcd0b3b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:52.731387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353e182ee28440d18416864adbcd0b3b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:52.751154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353e182ee28440d18416864adbcd0b3b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:52.762099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1022/2082] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:52 2022] DELETE /identity/v3/users/353e182ee28440d18416864adbcd0b3b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:12:52.763877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:52.764360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:52.764441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:52.792330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:52.793692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:52.842054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:52.875133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1061/2083] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:52.888345 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:52.889811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:52.948759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d6dcd4-a25d-43ce-890f-683943a9ae65 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:52.952438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:52.953856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:52.954148 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:52.954423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:52.963775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:52.965248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:52.975100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1023/2084] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:12:53.470035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:53.472374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:53.486236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256fc3af-bf76-4ce9-9e23-69ef6caa15a9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:53.494161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:53.494961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:53.495255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:53.495537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] PATH_INFO: `/v3/projects/beeecdb46ebc4ffc8e8230be3f0c5dae` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:53.496810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beeecdb46ebc4ffc8e8230be3f0c5dae)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:53.498955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:53.741843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beeecdb46ebc4ffc8e8230be3f0c5dae', 'request_id': 'req-742e6fbf-a725-4b46-a536-e338575cbf26'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:53.816357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beeecdb46ebc4ffc8e8230be3f0c5dae', 'request_id': 'req-742e6fbf-a725-4b46-a536-e338575cbf26'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:53.823935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beeecdb46ebc4ffc8e8230be3f0c5dae', 'request_id': 'req-742e6fbf-a725-4b46-a536-e338575cbf26'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:53.836757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] The token cache is being invalidate because project beeecdb46ebc4ffc8e8230be3f0c5dae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:53.837234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:53.837893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-742e6fbf-a725-4b46-a536-e338575cbf26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:53.839433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1062/2085] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:53 2022] DELETE /identity/v3/projects/beeecdb46ebc4ffc8e8230be3f0c5dae => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:57.274516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:57.276031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:57.289315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b392bf6-64c1-4417-bc45-968db4a2f0c0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:57.346615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:57.347185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:57.347471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:57.347751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] PATH_INFO: `/v3/users/e3227dfdb4c4453291f640e418ebebae` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:57.348948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3227dfdb4c4453291f640e418ebebae)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:57.350951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:57.374587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:57.572616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3227dfdb4c4453291f640e418ebebae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:57.580238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3227dfdb4c4453291f640e418ebebae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:57.690716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3227dfdb4c4453291f640e418ebebae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:57.691846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3227dfdb4c4453291f640e418ebebae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:57.722392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3227dfdb4c4453291f640e418ebebae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:57.732943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1024/2086] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:57 2022] DELETE /identity/v3/users/e3227dfdb4c4453291f640e418ebebae => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:57.738895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:57.739315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:57.739526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:57.759468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:12:57.760699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:57.850902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1063/2087] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:12:57.861016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:57.863378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:57.935209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eff18e-e5c0-47a5-a559-7fa0ba157c94 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:57.941192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:57.942089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:57.942584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:57.943203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:57.959939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:57.959939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:57.966514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1025/2088] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:12:58.561646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:58.564745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:12:58.587354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae5f8c-7e07-4cd9-b38a-73937051fb09 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:58.593415 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:58.594250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:58.598051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:58.598051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] PATH_INFO: `/v3/projects/093b02e628a944929ab40fcf2bc07b25` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:58.598051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=093b02e628a944929ab40fcf2bc07b25)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:58.599370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:58.770852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '093b02e628a944929ab40fcf2bc07b25', 'request_id': 'req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:58.873382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '093b02e628a944929ab40fcf2bc07b25', 'request_id': 'req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:58.888562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '093b02e628a944929ab40fcf2bc07b25', 'request_id': 'req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:58.894687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:58.904828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] The token cache is being invalidate because project 093b02e628a944929ab40fcf2bc07b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:12:58.905443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:58.907251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:58.909964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1064/2089] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:12:58 2022] DELETE /identity/v3/projects/093b02e628a944929ab40fcf2bc07b25 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:12:58.913449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:12:58.925346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1b80-8670-41cd-a35d-5be10ce30065 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:12:59.010088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:12:59.010766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:12:59.011048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:12:59.011325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] PATH_INFO: `/v3/users/62d85a3358ea4f26be9e8d57ab728f6c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:12:59.012459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d85a3358ea4f26be9e8d57ab728f6c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:12:59.015161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:12:59.274888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:12:59.277619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d85a3358ea4f26be9e8d57ab728f6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:59.285555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d85a3358ea4f26be9e8d57ab728f6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:59.425262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d85a3358ea4f26be9e8d57ab728f6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:59.426197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d85a3358ea4f26be9e8d57ab728f6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:59.445379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d85a3358ea4f26be9e8d57ab728f6c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:12:59.452507 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1026/2090] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:12:58 2022] DELETE /identity/v3/users/62d85a3358ea4f26be9e8d57ab728f6c => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:00.058862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:00.060365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:00.076377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a90174-6662-47a2-b5f6-1fe8eb4a8ba9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:00.154925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:00.155566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:00.155848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:00.156310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] PATH_INFO: `/v3/projects/ba8582f36aee4e579b408921af9061fa` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:00.158200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8582f36aee4e579b408921af9061fa)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:00.160126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:00.308163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8582f36aee4e579b408921af9061fa', 'request_id': 'req-639745bc-3612-4ea9-98dc-b0116b9f57c5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.420138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8582f36aee4e579b408921af9061fa', 'request_id': 'req-639745bc-3612-4ea9-98dc-b0116b9f57c5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.428223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8582f36aee4e579b408921af9061fa', 'request_id': 'req-639745bc-3612-4ea9-98dc-b0116b9f57c5'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.439456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] The token cache is being invalidate because project ba8582f36aee4e579b408921af9061fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:00.439748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.440398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.441934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1065/2091] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:00 2022] DELETE /identity/v3/projects/ba8582f36aee4e579b408921af9061fa => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:00.445881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:00.447623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:00.461001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9302c6c3-64ba-407f-b809-aa785968b819 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:00.520924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:00.521561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:00.521879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:00.522163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] PATH_INFO: `/v3/users/1a18dda92c12472f857c511be82526d8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:00.523684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a18dda92c12472f857c511be82526d8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:00.525623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:00.827342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a18dda92c12472f857c511be82526d8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.835338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a18dda92c12472f857c511be82526d8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.913118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a18dda92c12472f857c511be82526d8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.913820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a18dda92c12472f857c511be82526d8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.927376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8d08f54c-b774-445f-8a52-b852c4da0d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a18dda92c12472f857c511be82526d8'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:00.933488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1027/2092] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:00 2022] DELETE /identity/v3/users/1a18dda92c12472f857c511be82526d8 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:01.461265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:01.462942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:01.480093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639745bc-3612-4ea9-98dc-b0116b9f57c5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:01.559789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:01.567162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:01.567162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:01.567162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] PATH_INFO: `/v3/projects/edbbf7fcf85644bfa4c241e61f3ace18` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:01.567162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edbbf7fcf85644bfa4c241e61f3ace18)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:01.567162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:01.683984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:01.700580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edbbf7fcf85644bfa4c241e61f3ace18', 'request_id': 'req-db1e08e0-4dbf-4faa-adf2-b18398b40492'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:01.797194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edbbf7fcf85644bfa4c241e61f3ace18', 'request_id': 'req-db1e08e0-4dbf-4faa-adf2-b18398b40492'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:01.804516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edbbf7fcf85644bfa4c241e61f3ace18', 'request_id': 'req-db1e08e0-4dbf-4faa-adf2-b18398b40492'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:01.816121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] The token cache is being invalidate because project edbbf7fcf85644bfa4c241e61f3ace18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:01.816410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:01.817086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:01.818711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1066/2093] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:01 2022] DELETE /identity/v3/projects/edbbf7fcf85644bfa4c241e61f3ace18 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:07.787097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b2a4-3de0-42d4-84db-2e0b47c6b879 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:07.788217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b2a4-3de0-42d4-84db-2e0b47c6b879 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:07.788520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b2a4-3de0-42d4-84db-2e0b47c6b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:07.810750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2b30b2a4-3de0-42d4-84db-2e0b47c6b879 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:07.812049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b30b2a4-3de0-42d4-84db-2e0b47c6b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:07.849126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:07.870350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1028/2094] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:13:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:07.912799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:07.914223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:07.992571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1e08e0-4dbf-4faa-adf2-b18398b40492 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:08.000607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd94f515-81e3-4423-b7cc-2a3cb6e8ab86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:08.012773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f515-81e3-4423-b7cc-2a3cb6e8ab86 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:08.012773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f515-81e3-4423-b7cc-2a3cb6e8ab86 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:08.012773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd94f515-81e3-4423-b7cc-2a3cb6e8ab86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:08.017215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f515-81e3-4423-b7cc-2a3cb6e8ab86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:08.022417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd94f515-81e3-4423-b7cc-2a3cb6e8ab86 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:08.029019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1067/2095] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:13:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:13:17.940276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:18.026016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=6,keystone:SELECT=46,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:25.714441 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:25.717480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:25.717480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:25.740551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] MFA Rules not processed for user `76ea3ed3bb2b412093004c7e0f49c7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:25.741940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:25.834550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1029/2096] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:26.393699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54523d-7deb-49a1-a0b1-106b5409a30b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:26.394244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54523d-7deb-49a1-a0b1-106b5409a30b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:26.394354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54523d-7deb-49a1-a0b1-106b5409a30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:26.417841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5e54523d-7deb-49a1-a0b1-106b5409a30b None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:26.419086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5e54523d-7deb-49a1-a0b1-106b5409a30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:26.446458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:26.448174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1068/2097] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 30 01:13:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:26.453977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e96789-a9fc-43c3-8d15-78c53676158e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:26.460842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ce9cc-6be6-4f38-8ef0-699180b2b71a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:26.461557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ce9cc-6be6-4f38-8ef0-699180b2b71a service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:26.462092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ce9cc-6be6-4f38-8ef0-699180b2b71a service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:26.462576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ce9cc-6be6-4f38-8ef0-699180b2b71a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:26.480615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ce9cc-6be6-4f38-8ef0-699180b2b71a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:26.480771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53ce9cc-6be6-4f38-8ef0-699180b2b71a service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:26.498714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1030/2098] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:13:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:13:29.508284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:29.508785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:29.508876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:29.529140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] MFA Rules not processed for user `942d175e7e2742728e66fea84c851038`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:29.530377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:29.588080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1069/2099] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:31.502376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6f2f2-8fba-4a36-a099-e20c640452c2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:31.502877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6f2f2-8fba-4a36-a099-e20c640452c2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:31.503095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6f2f2-8fba-4a36-a099-e20c640452c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:31.537101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fbb6f2f2-8fba-4a36-a099-e20c640452c2 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:31.538634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fbb6f2f2-8fba-4a36-a099-e20c640452c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:31.547375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1031/2100] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:13:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:31.561502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:31.581127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ff568-5fe6-45f6-b395-aff8770541c0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:31.590968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912c0e52-6a80-451c-85de-55f5b6ecaf05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:31.591605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0e52-6a80-451c-85de-55f5b6ecaf05 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:31.592177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0e52-6a80-451c-85de-55f5b6ecaf05 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:31.592260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912c0e52-6a80-451c-85de-55f5b6ecaf05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:31.607608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c0e52-6a80-451c-85de-55f5b6ecaf05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:31.608088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912c0e52-6a80-451c-85de-55f5b6ecaf05 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:31.618965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1070/2101] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:13:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:13:34.141980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab91dc8-7c99-4cce-a9f9-4593ddccd31a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:34.142945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab91dc8-7c99-4cce-a9f9-4593ddccd31a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:34.143255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab91dc8-7c99-4cce-a9f9-4593ddccd31a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:34.145194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1032/2102] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:13:34.164971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:34.166118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:34.166561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:34.199807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] MFA Rules not processed for user `38d22a2e2f374aa6bba35838ebde3b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:34.201176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:34.337776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1071/2103] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:13:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:34.353647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab91dc8-7c99-4cce-a9f9-4593ddccd31a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:34.355564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aab91dc8-7c99-4cce-a9f9-4593ddccd31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:34.375336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:34.377173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:34.392196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab91dc8-7c99-4cce-a9f9-4593ddccd31a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:34.408022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:34.409352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:34.410894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:34.411632 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:34.424681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:34.427420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:34.441366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1033/2104] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:34.488723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144fa747-8288-4f2f-896f-1d68a86618d6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:34.509576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:34.510304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:34.510586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:34.510916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] PATH_INFO: `/v3/users/1b225b55d18740c2833cade52edc556e` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:34.512132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b225b55d18740c2833cade52edc556e)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:34.514678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:34.584249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:35.096236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b225b55d18740c2833cade52edc556e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:35.111343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b225b55d18740c2833cade52edc556e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:35.111343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b225b55d18740c2833cade52edc556e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:35.111343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b225b55d18740c2833cade52edc556e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:35.138392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-954e0df5-1556-4f89-ab0f-aaf1c9c76ab7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b225b55d18740c2833cade52edc556e'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:35.228825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1072/2105] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:34 2022] DELETE /identity/v3/users/1b225b55d18740c2833cade52edc556e => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:35.828370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:35.844404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a5af51-fd44-4c9d-a66e-3d30fc743ad5 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:35.945349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:35.947929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:35.947929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:35.947929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] PATH_INFO: `/v3/projects/43f140e0f58f42ae96e16f80b480621c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:35.949025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43f140e0f58f42ae96e16f80b480621c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:35.951091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:36.105011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43f140e0f58f42ae96e16f80b480621c', 'request_id': 'req-c0402a3c-f1b6-47d5-974e-3c505179ddc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:36.114413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43f140e0f58f42ae96e16f80b480621c', 'request_id': 'req-c0402a3c-f1b6-47d5-974e-3c505179ddc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:36.124410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43f140e0f58f42ae96e16f80b480621c', 'request_id': 'req-c0402a3c-f1b6-47d5-974e-3c505179ddc2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:36.202902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] The token cache is being invalidate because project 43f140e0f58f42ae96e16f80b480621c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:36.203187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:36.214335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c0402a3c-f1b6-47d5-974e-3c505179ddc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:36.219615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1034/2106] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:35 2022] DELETE /identity/v3/projects/43f140e0f58f42ae96e16f80b480621c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:36.221028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:36.221329 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:36.221616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:36.275567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:36.279246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:36.385682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1073/2107] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:36.393753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:36.394042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:36.394991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:36.407407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:36.422272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:36.425415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:36.432553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1035/2108] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:36.437310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:36.451298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f585c-0fef-4c63-b846-2662ebf4e92b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:36.457992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:36.464001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:36.464001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:36.464001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:36.464001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:36.464001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:36.464001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:36.485550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1074/2109] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:36.488778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:36.510665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993e32d-e1e0-4adb-a077-843fc9dc41c5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:36.510665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:36.510665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:36.510665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:36.511146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:36.511713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:36.513850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:36.708814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1036/2110] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:36 2022] POST /identity/v3/projects => generated 385 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:36.712704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:36.729212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21e336-aff8-4aef-8921-f489f63e4a4d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:36.853429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:36.853554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:36.853823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:36.854895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:36.855950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:36.857964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:36.959083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1075/2111] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:36 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:36.963244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:36.982198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51654dd-e11f-4f85-a019-7f87be6ede9a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:36.991969 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:36.992595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:36.993516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:36.993995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:36.994824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:36.997328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:37.010957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1037/2112] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:37.015032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:37.032050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48146bf-b4bd-4db6-8bd7-8c164c12e5ad admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:37.038712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4983da9-8e9c-4230-ad03-eb519423a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:37.040921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983da9-8e9c-4230-ad03-eb519423a754 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:37.040921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983da9-8e9c-4230-ad03-eb519423a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:37.040921 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4983da9-8e9c-4230-ad03-eb519423a754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:37.041558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4983da9-8e9c-4230-ad03-eb519423a754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:37.043365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4983da9-8e9c-4230-ad03-eb519423a754 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:37.060582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1076/2113] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:37.064279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:37.077642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f15daab-0446-4ce2-a3de-b58c7956c11d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:37.083122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:37.084691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:37.085196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:37.085815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] PATH_INFO: `/v3/projects/7db4c75cc0034358ad5aa5c681aed504/users/9a1859e3ff8441eb94b6c93ebbfd3a65/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:37.135996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7db4c75cc0034358ad5aa5c681aed504, user_id=9a1859e3ff8441eb94b6c93ebbfd3a65, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:37.139390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:37.245641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1038/2114] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:37 2022] PUT /identity/v3/projects/7db4c75cc0034358ad5aa5c681aed504/users/9a1859e3ff8441eb94b6c93ebbfd3a65/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:37.257087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:37.257379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:37.257660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:37.279678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:37.347947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] MFA Rules not processed for user `9a1859e3ff8441eb94b6c93ebbfd3a65`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:37.349486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:37.499566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1077/2115] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:37.527888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:37.528752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:37.669448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591b45b-9715-43cf-a53b-423362866d5f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:37.682785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88196f63-9a70-4d85-88cb-0e741f577e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:37.685503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88196f63-9a70-4d85-88cb-0e741f577e50 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:37.686358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88196f63-9a70-4d85-88cb-0e741f577e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:37.687444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88196f63-9a70-4d85-88cb-0e741f577e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:37.711647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88196f63-9a70-4d85-88cb-0e741f577e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:37.716660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88196f63-9a70-4d85-88cb-0e741f577e50 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:37.764533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1039/2116] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:37.956917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:37.962237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:38.102643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b55f8-a0a1-4856-a881-36fd31f3f6fa None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:38.107760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:38.108638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:38.109348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:38.109348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] PATH_INFO: `/v3/users/a167b5065cc54f0186af5596408b234c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:38.111458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a167b5065cc54f0186af5596408b234c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:38.112735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:38.362651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a167b5065cc54f0186af5596408b234c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:38.374739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a167b5065cc54f0186af5596408b234c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:38.375779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a167b5065cc54f0186af5596408b234c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:38.377169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a167b5065cc54f0186af5596408b234c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:38.399800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a167b5065cc54f0186af5596408b234c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:38.533438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1078/2117] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:37 2022] DELETE /identity/v3/users/a167b5065cc54f0186af5596408b234c => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:38.538308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:38.538898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:38.538898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:38.573567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:38.575136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:38.665940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1040/2118] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:38.679652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:38.681448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:38.791508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e281b-2b2c-4628-ae3b-1a96bc19bcd9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:38.803756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:38.810907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:38.810907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:38.810907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:38.826257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:38.827924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:38.845274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1079/2119] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:39.015336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:39.411891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:39.411891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:39.422433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeb5d02-be81-4c5e-a5ca-4b698265ca9c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:39.444395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:39.445167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:39.445479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:39.445785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] PATH_INFO: `/v3/projects/bec70e149d0542bc8c1bcc9f29083f46` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:39.448206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bec70e149d0542bc8c1bcc9f29083f46)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:39.450806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:39.601500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:39.611984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bec70e149d0542bc8c1bcc9f29083f46', 'request_id': 'req-317693fb-ea7b-4263-abe7-ea5946afbb88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:39.620574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bec70e149d0542bc8c1bcc9f29083f46', 'request_id': 'req-317693fb-ea7b-4263-abe7-ea5946afbb88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:39.629333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bec70e149d0542bc8c1bcc9f29083f46', 'request_id': 'req-317693fb-ea7b-4263-abe7-ea5946afbb88'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:39.747329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] The token cache is being invalidate because project bec70e149d0542bc8c1bcc9f29083f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:39.747646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:39.748452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:39.750508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1041/2120] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:39 2022] DELETE /identity/v3/projects/bec70e149d0542bc8c1bcc9f29083f46 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:39.755304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:39.765997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786735c2-1d3e-4b1f-b05e-1fb9381b149f service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:39.840423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:39.840423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:39.840423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:39.840815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] PATH_INFO: `/v3/users/589f69ef16d545f8bda4559bbbeeae6c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:39.841999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=589f69ef16d545f8bda4559bbbeeae6c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:39.844136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:40.447649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '589f69ef16d545f8bda4559bbbeeae6c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:40.473132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '589f69ef16d545f8bda4559bbbeeae6c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:40.480776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '589f69ef16d545f8bda4559bbbeeae6c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:40.480776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '589f69ef16d545f8bda4559bbbeeae6c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:40.541312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '589f69ef16d545f8bda4559bbbeeae6c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:40.684318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1080/2121] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:39 2022] DELETE /identity/v3/users/589f69ef16d545f8bda4559bbbeeae6c => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:41.508316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:41.509962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:41.528398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317693fb-ea7b-4263-abe7-ea5946afbb88 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:41.569628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:41.571243 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:41.633525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:41.634232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:41.634513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:41.634793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] PATH_INFO: `/v3/projects/2db63e2c839c4583ad050d2243656a45` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:41.636333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2db63e2c839c4583ad050d2243656a45)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:41.638293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:41.703370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:41.738050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d730cb-328f-4973-be93-893c18b2a1b3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:41.743405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:41.744149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:41.744649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:41.745018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] PATH_INFO: `/v3/users/9a1859e3ff8441eb94b6c93ebbfd3a65` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:41.746512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a1859e3ff8441eb94b6c93ebbfd3a65)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:41.761776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:41.815075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2db63e2c839c4583ad050d2243656a45', 'request_id': 'req-8809b5f9-4948-466f-a1b8-2035a2dabb9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:41.827803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2db63e2c839c4583ad050d2243656a45', 'request_id': 'req-8809b5f9-4948-466f-a1b8-2035a2dabb9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:41.840129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2db63e2c839c4583ad050d2243656a45', 'request_id': 'req-8809b5f9-4948-466f-a1b8-2035a2dabb9a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:41.958476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] The token cache is being invalidate because project 2db63e2c839c4583ad050d2243656a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:41.958802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:41.961027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8809b5f9-4948-466f-a1b8-2035a2dabb9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:41.962824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1042/2122] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:41 2022] DELETE /identity/v3/projects/2db63e2c839c4583ad050d2243656a45 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:41.968111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f641-52e3-41b0-ade6-9fa6ff663a44 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:41.968393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f641-52e3-41b0-ade6-9fa6ff663a44 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:41.968668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f641-52e3-41b0-ade6-9fa6ff663a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:41.995270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0753f641-52e3-41b0-ade6-9fa6ff663a44 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:41.996395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0753f641-52e3-41b0-ade6-9fa6ff663a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:42.062199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:42.084982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1043/2123] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:42.091603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a1859e3ff8441eb94b6c93ebbfd3a65'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:42.095034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.095300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.095862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.110081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a1859e3ff8441eb94b6c93ebbfd3a65'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:42.113131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a1859e3ff8441eb94b6c93ebbfd3a65'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:42.114421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a1859e3ff8441eb94b6c93ebbfd3a65'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:42.126082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:42.128052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:42.141577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1044/2124] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:42.151385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fb4ace68-64c9-4154-9a14-7b2ff6ac4979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a1859e3ff8441eb94b6c93ebbfd3a65'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:42.155810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.171769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b5c77d-7ede-4fb2-863a-5af423b6610b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.177966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.178848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.179136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.179504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.180891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:42.181211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.183171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.197778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1045/2125] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:42.201782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.217639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6e0f9-5949-46fa-b95b-1499eb38ebff admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.224971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.226420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.226789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.227173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.227914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.230296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.243650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1081/2126] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:41 2022] DELETE /identity/v3/users/9a1859e3ff8441eb94b6c93ebbfd3a65 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:42.249100 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.249284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.254084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.286853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:42.288342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:42.357910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1046/2127] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:42 2022] POST /identity/v3/projects => generated 369 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:13:42.360737 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.438031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1082/2128] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:42.451391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.453367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:42.494724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c5aea1-b67a-470c-a40a-5c02060ebacb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.502340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.503647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.506703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.507174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.507948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.516813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.577215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be074433-17c7-467b-9e4a-4229883abfbd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.596297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1047/2129] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:42 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:42.599264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.631409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29f5cc-66e2-47f0-bfb6-b6018ef34dc2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.638359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.639244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.639685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.640117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.640960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.642787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.667217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1048/2130] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:13:42.669019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.679177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c3115-ec04-49e3-8d48-ab0f30883700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.680124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c3115-ec04-49e3-8d48-ab0f30883700 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.680423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c3115-ec04-49e3-8d48-ab0f30883700 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.680701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c3115-ec04-49e3-8d48-ab0f30883700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.696444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a70b35-2408-4fa3-8afe-6f6d2232744d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.704707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c3115-ec04-49e3-8d48-ab0f30883700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.707750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c3115-ec04-49e3-8d48-ab0f30883700 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.709252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.709947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.710238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.710601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.711328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.713164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.732010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1049/2131] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:42.735088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1083/2132] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:42.736093 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:42.740641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:42.751211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a0212-0504-4467-aed3-5af0e7e66759 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:42.758379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:42.759419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.760390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.760904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] PATH_INFO: `/v3/projects/01541666b0b2433b8551cbb6b279c081/users/1ef365501dcc44b7969e5921461ac911/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.783325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01541666b0b2433b8551cbb6b279c081, user_id=1ef365501dcc44b7969e5921461ac911, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:42.788426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:42.887255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1050/2133] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:42 2022] PUT /identity/v3/projects/01541666b0b2433b8551cbb6b279c081/users/1ef365501dcc44b7969e5921461ac911/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:42.904426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:42.905054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:42.909564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:42.946578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] MFA Rules not processed for user `1ef365501dcc44b7969e5921461ac911`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:42.947943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:43.114858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1084/2134] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:43.119057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:43.250856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:43.267346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc267f-43b1-4cda-8b97-ca7b1fa29321 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:43.274995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:43.275892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:43.276424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:43.276902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:43.277779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:43.280102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:43.438259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1051/2135] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:43 2022] POST /identity/v3/projects => generated 369 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:13:43.445418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:43.473019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd56b8c-a967-4812-8100-8ac6e67031cc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:43.571826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:43.572643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:43.572918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:43.573208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:43.573914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:43.575881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:43.609572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:43.613944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:43.640965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47df6b1-9bf7-484d-9b93-0c71d80694b8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:43.649493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:43.656683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:43.656683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:43.656683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] PATH_INFO: `/v3/projects/7db4c75cc0034358ad5aa5c681aed504` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:43.656683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7db4c75cc0034358ad5aa5c681aed504)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:43.656683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:43.690316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1085/2136] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:43 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:43.697105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:43.714887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4091ae9e-fc09-41e2-8b56-f78a62721627 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:43.721077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:43.721715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:43.722067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:43.722476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:43.723140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:43.724966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:43.740173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1086/2137] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:43.744439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:43.760388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167b1a90-36aa-404a-bb92-b091b3313379 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:43.767662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:43.768357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:43.768693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:43.769087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] PATH_INFO: `/v3/projects/ca7893ec111940958505ded5ce1b08a9/users/140444c16bde459a994a02d8a58ff5f8/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:43.795808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7893ec111940958505ded5ce1b08a9, user_id=140444c16bde459a994a02d8a58ff5f8, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:43.799476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:43.880652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:43.889422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1087/2138] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:43 2022] PUT /identity/v3/projects/ca7893ec111940958505ded5ce1b08a9/users/140444c16bde459a994a02d8a58ff5f8/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:43.891722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:43.899557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7db4c75cc0034358ad5aa5c681aed504', 'request_id': 'req-01c3b79e-349d-487f-bd8e-a54f3dbc0321'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:43.908475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7db4c75cc0034358ad5aa5c681aed504', 'request_id': 'req-01c3b79e-349d-487f-bd8e-a54f3dbc0321'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:43.918180 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7db4c75cc0034358ad5aa5c681aed504', 'request_id': 'req-01c3b79e-349d-487f-bd8e-a54f3dbc0321'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:44.016436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88103748-53bb-4aad-b3ae-2985b29ef3d0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:44.020039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] The token cache is being invalidate because project 7db4c75cc0034358ad5aa5c681aed504 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:44.020342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:44.021020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-01c3b79e-349d-487f-bd8e-a54f3dbc0321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:44.023559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1052/2139] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:43 2022] DELETE /identity/v3/projects/7db4c75cc0034358ad5aa5c681aed504 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:44.028542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772e27-f7f9-42d8-851c-cb28cbc8efca None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.028816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772e27-f7f9-42d8-851c-cb28cbc8efca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.029098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772e27-f7f9-42d8-851c-cb28cbc8efca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.062336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a6772e27-f7f9-42d8-851c-cb28cbc8efca None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:44.065314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a6772e27-f7f9-42d8-851c-cb28cbc8efca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:44.099574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:44.100838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.101135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.101520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.103101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:44.104857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:44.124117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1088/2140] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:44.125097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:44.127374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:44.145103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b7643-5814-4276-8874-9ef883cdce58 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:44.151039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154e3c0-ef4a-4dd7-85ac-e0afeabcf9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:44.151810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e3c0-ef4a-4dd7-85ac-e0afeabcf9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.152172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e3c0-ef4a-4dd7-85ac-e0afeabcf9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.152597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154e3c0-ef4a-4dd7-85ac-e0afeabcf9cd admin admin] PATH_INFO: `/v3/projects/ca7893ec111940958505ded5ce1b08a9/users/140444c16bde459a994a02d8a58ff5f8/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.155302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154e3c0-ef4a-4dd7-85ac-e0afeabcf9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca7893ec111940958505ded5ce1b08a9, user_id=140444c16bde459a994a02d8a58ff5f8, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:44.158538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154e3c0-ef4a-4dd7-85ac-e0afeabcf9cd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:44.176445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1089/2141] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:44 2022] PUT /identity/v3/projects/ca7893ec111940958505ded5ce1b08a9/users/140444c16bde459a994a02d8a58ff5f8/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:44.186171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.186380 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.186574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.249719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] MFA Rules not processed for user `140444c16bde459a994a02d8a58ff5f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:44.253352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:44.267547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1053/2142] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:13:44.271106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.271411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.271694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.348889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:44.348889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:44.438725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1090/2143] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:44.446600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:44.446600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:44.499638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:44.517860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1054/2144] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:13:44.522809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:44.548785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c64e6-efb9-45c6-8e5b-977b9059d9d1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:44.582529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f981227b-31b4-41e4-b818-879cc8b9f52b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:44.582964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:44.584319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.584319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.584319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.584942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:44.585256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:44.586983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:44.590966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:44.591955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:44.592591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.592903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.593181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.615487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1055/2145] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:44.616059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:44.616059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:44.618001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:44.626609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1091/2146] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:44.640401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c522a29-d583-4c89-b2a8-1e0d3f78d412 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:44.654568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:44.655809 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:44.656359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:44.656878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:44.658831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:44.661177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:44.971839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1056/2147] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:44 2022] POST /identity/v3/projects => generated 365 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:13:44.972368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:44.980117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:44.981930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:44.982249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9012141-3914-4693-b63d-d7be08e285d8 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.041436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b579905-b7ef-41ed-8ffc-a1fbba1ed93c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.044755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.045469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.045659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.045962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.046118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.046223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.046399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.046702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.046999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.048489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:45.049064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.117417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.119033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.131908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1057/2148] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:45.207519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1092/2149] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:44 2022] POST /identity/v3/users => generated 283 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:13:45.208810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:45.219637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f84e7-6998-4dfc-8cb2-5fbd2aef656f service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.224509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.225106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.225390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.225669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.226340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.227960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.243756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1058/2150] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:45.244915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:45.257456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097f49b8-c473-41ba-a172-33b28e5912ea admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.262505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bfbd90-516a-4a59-b4c5-df87f40e4065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.263062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfbd90-516a-4a59-b4c5-df87f40e4065 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.263346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfbd90-516a-4a59-b4c5-df87f40e4065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.263622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfbd90-516a-4a59-b4c5-df87f40e4065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.264515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfbd90-516a-4a59-b4c5-df87f40e4065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.266165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bfbd90-516a-4a59-b4c5-df87f40e4065 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.276630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1093/2151] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:13:45.280934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:45.291579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a604550a-0f55-4c8b-9ddf-de6d2572ef2f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.296443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.297096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.297380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.297663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] PATH_INFO: `/v3/projects/273553011da245b5b18e2c606124ed1a/users/aaf5709310884616acc4beebbdb3cf4b/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.313589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=273553011da245b5b18e2c606124ed1a, user_id=aaf5709310884616acc4beebbdb3cf4b, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.316526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.435531 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1059/2152] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:45 2022] PUT /identity/v3/projects/273553011da245b5b18e2c606124ed1a/users/aaf5709310884616acc4beebbdb3cf4b/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:45.447019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.447323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.447606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.505314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] MFA Rules not processed for user `aaf5709310884616acc4beebbdb3cf4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:45.509444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:45.589949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1094/2153] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:45.602799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:45.696564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2c81ed-be59-4466-9262-21d5071dda8f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:45.710386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d809e4-0022-4bc2-a0c4-aa3240f83255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:45.711035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d809e4-0022-4bc2-a0c4-aa3240f83255 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:45.711320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d809e4-0022-4bc2-a0c4-aa3240f83255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:45.711597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d809e4-0022-4bc2-a0c4-aa3240f83255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:45.720425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:45.728118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d809e4-0022-4bc2-a0c4-aa3240f83255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:45.732972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d809e4-0022-4bc2-a0c4-aa3240f83255 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:45.754259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1060/2154] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:50.685104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:50.734436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:50.780934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb9c9e3-39c5-4a07-987d-4b49a07b6c10 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:50.786930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:50.787588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:50.788147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:50.788670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] PATH_INFO: `/v3/users/1ef365501dcc44b7969e5921461ac911` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:50.790423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef365501dcc44b7969e5921461ac911)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:50.792537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:51.000598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef365501dcc44b7969e5921461ac911'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:51.010114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef365501dcc44b7969e5921461ac911'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:51.010878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef365501dcc44b7969e5921461ac911'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:51.011495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef365501dcc44b7969e5921461ac911'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:51.028797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef365501dcc44b7969e5921461ac911'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:51.091463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1095/2155] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:50 2022] DELETE /identity/v3/users/1ef365501dcc44b7969e5921461ac911 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:51.096192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:51.098051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:51.098606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:51.123259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:51.124558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:51.188274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1061/2156] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:51.198294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:51.199740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:51.258490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc061ae3-8637-460e-8220-9f261cd111e5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:51.261985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:51.262588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:51.262871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:51.263148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:51.273325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:51.274872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:51.285173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1096/2157] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:51.834568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:51.836664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:51.856633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f7fff1-dba9-468d-a272-8aff9317ba43 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:51.863041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:51.863789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:51.864208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:51.864479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] PATH_INFO: `/v3/projects/01541666b0b2433b8551cbb6b279c081` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:51.865699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01541666b0b2433b8551cbb6b279c081)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:51.867638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:52.009675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01541666b0b2433b8551cbb6b279c081', 'request_id': 'req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.019326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01541666b0b2433b8551cbb6b279c081', 'request_id': 'req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.025765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01541666b0b2433b8551cbb6b279c081', 'request_id': 'req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.133200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] The token cache is being invalidate because project 01541666b0b2433b8551cbb6b279c081 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:52.133948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.134617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.136230 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1062/2158] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:51 2022] DELETE /identity/v3/projects/01541666b0b2433b8551cbb6b279c081 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:52.140350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:52.150720 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db909b72-2be2-4177-8ee4-890acd6b2f2c service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:52.202421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:52.215402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:52.216230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:52.216599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:52.216951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] PATH_INFO: `/v3/users/140444c16bde459a994a02d8a58ff5f8` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:52.218323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=140444c16bde459a994a02d8a58ff5f8)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:52.220359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:52.395832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '140444c16bde459a994a02d8a58ff5f8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.404168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '140444c16bde459a994a02d8a58ff5f8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.404897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '140444c16bde459a994a02d8a58ff5f8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.405518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '140444c16bde459a994a02d8a58ff5f8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.419692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '140444c16bde459a994a02d8a58ff5f8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:52.496414 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1097/2159] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:52 2022] DELETE /identity/v3/users/140444c16bde459a994a02d8a58ff5f8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:52.989003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:52.990766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.006954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b91f5f2-c7eb-46c5-922a-d2322bc7b247 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.074534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.075451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.075730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.076002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] PATH_INFO: `/v3/projects/ca7893ec111940958505ded5ce1b08a9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.077124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca7893ec111940958505ded5ce1b08a9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.079571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.103613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:53.139405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.140862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.226047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca7893ec111940958505ded5ce1b08a9', 'request_id': 'req-339e8404-cb79-4a8a-98ff-98aa1f7d6539'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.233622 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca7893ec111940958505ded5ce1b08a9', 'request_id': 'req-339e8404-cb79-4a8a-98ff-98aa1f7d6539'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.245541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ae8db-9cb5-4a97-a77f-4c09728da132 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.247537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca7893ec111940958505ded5ce1b08a9', 'request_id': 'req-339e8404-cb79-4a8a-98ff-98aa1f7d6539'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.318411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.318989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.319467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.319820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.321618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.335628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.337290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3179e0-797f-4f67-82e8-dcd69b492bb8 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.346174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:53.353480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1098/2160] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:53.382412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] The token cache is being invalidate because project ca7893ec111940958505ded5ce1b08a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:53.388920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.388920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-339e8404-cb79-4a8a-98ff-98aa1f7d6539 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:53.388920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1063/2161] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:52 2022] DELETE /identity/v3/projects/ca7893ec111940958505ded5ce1b08a9 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:53.391088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.391458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.391811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.417346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:53.423361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.503624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1099/2162] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:13:53.511946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.512250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.512563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.533790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:53.535934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:53.549780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1064/2163] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:53.553871 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.563819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009cc99f-183a-427a-af4b-dcefbb93c0ba None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.569213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.569867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.570150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.570451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.571083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:53.571386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.572989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.585498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1100/2164] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:53.590410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.602482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4e70b7-12f8-45b5-ae34-bee9d4244dbd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.608730 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.609714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.610038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.610332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.611207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.614603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.739775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1065/2165] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:53 2022] POST /identity/v3/projects => generated 379 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:53.743939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.754808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42fdbd-d892-4a36-9b73-e381523350f8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.818032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.818940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.819238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.819504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.820146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.822098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.934803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1101/2166] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:53 2022] POST /identity/v3/users => generated 290 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:53.938883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.949351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb8a32-1849-4bad-95a2-a45fd52456c3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.955110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.955838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.956678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.956878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.958136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.960904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:53.976027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1066/2167] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:53.978176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:53.989226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aab2034-c953-4eb4-bc05-9b5e6fa682f1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:53.994578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9391888-e692-4ed0-a25b-1ddf5f23f747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:53.995089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9391888-e692-4ed0-a25b-1ddf5f23f747 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:53.998992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9391888-e692-4ed0-a25b-1ddf5f23f747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:53.998992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9391888-e692-4ed0-a25b-1ddf5f23f747 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:53.998992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9391888-e692-4ed0-a25b-1ddf5f23f747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:53.998992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9391888-e692-4ed0-a25b-1ddf5f23f747 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.009161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1102/2168] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:54.013488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.024881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c8d87f-23c6-4167-8243-3a335e74c2de admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.029498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.030197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.030480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.030759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] PATH_INFO: `/v3/projects/7e45f2de355d420bba357f8a56483314/users/4769c755b54442c493057adabcc87637/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.047483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e45f2de355d420bba357f8a56483314, user_id=4769c755b54442c493057adabcc87637, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.050173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.145639 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1067/2169] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:54 2022] PUT /identity/v3/projects/7e45f2de355d420bba357f8a56483314/users/4769c755b54442c493057adabcc87637/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:54.157029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.157563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.157563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.188447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] MFA Rules not processed for user `4769c755b54442c493057adabcc87637`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:54.189697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:54.248325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:54.257945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1103/2170] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:54.262614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.328864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3e7619-d522-448c-b2a8-f9c7f8f3566c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.333788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.334908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.335193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.335473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.336122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.338396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.456203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1068/2171] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:54 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:54.460979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.482223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe52d9-2c08-4658-a656-4b6592191b3d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.548640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.549388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.549809 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.550212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.550937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.552892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.649094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1104/2172] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:13:54 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:54.653291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.665190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351a8ba-814d-484d-9f46-489eb7c17a4e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.672727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.673536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.673686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.674003 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.675900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.677529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.690318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1069/2173] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:54.696168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.717302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00d830-ce5c-45f3-beec-3e72e620cba0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.722678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.723361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.723640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.723915 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] PATH_INFO: `/v3/projects/4b80629cc66c49219032ad3e16b3ca3c/users/2a948936ad7a4636b85299a2eca6d602/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.741676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b80629cc66c49219032ad3e16b3ca3c, user_id=2a948936ad7a4636b85299a2eca6d602, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.744849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.877861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1105/2174] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:54 2022] PUT /identity/v3/projects/4b80629cc66c49219032ad3e16b3ca3c/users/2a948936ad7a4636b85299a2eca6d602/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:54.882598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.898892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff852c9a-27e5-4d2c-af31-74468905697d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.913878 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:54.965765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68b574-4753-402b-b05e-d23e2acdd5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:54.966380 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68b574-4753-402b-b05e-d23e2acdd5e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:54.966678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68b574-4753-402b-b05e-d23e2acdd5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:54.966931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68b574-4753-402b-b05e-d23e2acdd5e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:54.967566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68b574-4753-402b-b05e-d23e2acdd5e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:54.969145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68b574-4753-402b-b05e-d23e2acdd5e7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:54.979079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1070/2175] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:13:54.983391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:54.994783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82bec42-933e-44af-b6e9-087842d2cb6d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:54.999447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:55.000085 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:55.000369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:55.000641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] PATH_INFO: `/v3/projects/4b80629cc66c49219032ad3e16b3ca3c/users/2a948936ad7a4636b85299a2eca6d602/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:55.002622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b80629cc66c49219032ad3e16b3ca3c, user_id=2a948936ad7a4636b85299a2eca6d602, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:55.005881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:55.016190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1106/2176] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:13:54 2022] PUT /identity/v3/projects/4b80629cc66c49219032ad3e16b3ca3c/users/2a948936ad7a4636b85299a2eca6d602/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:13:55.026966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:55.027260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:55.027543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:55.070652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] MFA Rules not processed for user `2a948936ad7a4636b85299a2eca6d602`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:55.072006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:55.168828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1071/2177] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:55.185570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:55.187425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:55.251056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708cbced-f7f8-477e-b4dd-80b94d2a3875 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:55.255157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:55.255848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:55.256055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:55.256406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:55.266419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:55.267935 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:55.277736 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1107/2178] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:13:56.422666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:56.432750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a93e0-5887-4f9e-ab70-88595965dee0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:56.437631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a948936ad7a4636b85299a2eca6d602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b80629cc66c49219032ad3e16b3ca3c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:56.438316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:56.438605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:56.438883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:56.439570 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:56.439881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:56.442529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:56.455807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1072/2179] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:56.460462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:56.470237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e895d23-b85e-4f05-9fcd-53de96ae588d service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:56.475445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a948936ad7a4636b85299a2eca6d602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b80629cc66c49219032ad3e16b3ca3c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:56.476052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:56.476356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:56.478618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:56.478618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:56.479467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:56.599857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1108/2180] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:13:56 2022] POST /identity/v3/projects => generated 354 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:13:56.604203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:56.614837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd8b0fe-59d2-40cc-8184-80986b28ecb4 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:56.671537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a948936ad7a4636b85299a2eca6d602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b80629cc66c49219032ad3e16b3ca3c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:56.672215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:56.672501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:56.672778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:56.673433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:13:56.673751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:56.675950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6cd778-a81b-4121-8cce-f63c72b23458 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:56.692001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1073/2181] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:13:56.731786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:56.744099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241721bb-53bf-4d11-96b5-46650ff42e04 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:56.749076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a948936ad7a4636b85299a2eca6d602', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4b80629cc66c49219032ad3e16b3ca3c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:56.749780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:56.750068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:56.750343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] PATH_INFO: `/v3/projects/f691b2041b84422f87038a63fdc4f8c3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:56.751766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorizing `identity:delete_project(project_id=f691b2041b84422f87038a63fdc4f8c3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:56.754075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:56.760370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fb69d-1c1f-42b0-8217-a6a7c3914a10 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:56.760936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fb69d-1c1f-42b0-8217-a6a7c3914a10 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:56.761429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632fb69d-1c1f-42b0-8217-a6a7c3914a10 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:56.762949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1074/2182] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 01:13:56.770560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:56.770849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:56.771425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:56.787464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:56.794300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:56.796156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:56.811688 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:56.871171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1075/2183] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:13:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:13:56.872467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f691b2041b84422f87038a63fdc4f8c3', 'request_id': 'req-3e4fadcb-d565-4fcb-9cd2-861805edd257'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:56.881710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:56.883218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:56.945255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f691b2041b84422f87038a63fdc4f8c3', 'request_id': 'req-3e4fadcb-d565-4fcb-9cd2-861805edd257'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:56.953542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f691b2041b84422f87038a63fdc4f8c3', 'request_id': 'req-3e4fadcb-d565-4fcb-9cd2-861805edd257'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:56.963546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c335546-cc0a-4e9a-b7a6-2c85c83a230a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:56.965497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] The token cache is being invalidate because project f691b2041b84422f87038a63fdc4f8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:56.968950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:56.968950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3e4fadcb-d565-4fcb-9cd2-861805edd257 tempest-VolumeQuotaClassesTest-1672521995 tempest-VolumeQuotaClassesTest-1672521995-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:56.968950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1109/2184] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:56 2022] DELETE /identity/v3/projects/f691b2041b84422f87038a63fdc4f8c3 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:57.015396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:57.016007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.016290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.016560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.018342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:57.077635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:57.093584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:57.095164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba21c81-e17f-4d25-9e17-d99d08f4272c service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:57.110115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1076/2185] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:13:57.400271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc2d9-b026-43d6-977d-15fc82291bbe None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.400463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc2d9-b026-43d6-977d-15fc82291bbe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.400738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dadc2d9-b026-43d6-977d-15fc82291bbe None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.402126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1110/2186] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:13:57.407187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b94bd-345c-4cea-84e1-750d7a90b274 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.407483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b94bd-345c-4cea-84e1-750d7a90b274 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.407760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b94bd-345c-4cea-84e1-750d7a90b274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.429821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f51b94bd-345c-4cea-84e1-750d7a90b274 None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:57.431879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f51b94bd-345c-4cea-84e1-750d7a90b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:57.493477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1077/2187] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:13:57.506208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadc2d9-b026-43d6-977d-15fc82291bbe None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:57.508036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6dadc2d9-b026-43d6-977d-15fc82291bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:57.522168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dadc2d9-b026-43d6-977d-15fc82291bbe None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:57.525702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca7ef5-8f6c-422b-9c89-f1cf3bdddaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:57.526353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7ef5-8f6c-422b-9c89-f1cf3bdddaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.526822 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7ef5-8f6c-422b-9c89-f1cf3bdddaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.527059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca7ef5-8f6c-422b-9c89-f1cf3bdddaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.534264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca7ef5-8f6c-422b-9c89-f1cf3bdddaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:57.536043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca7ef5-8f6c-422b-9c89-f1cf3bdddaa2 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:57.543851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1111/2188] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:57.635355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef0bdb-7f80-4c88-be63-aa969c35a5cc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.635674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef0bdb-7f80-4c88-be63-aa969c35a5cc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.635959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef0bdb-7f80-4c88-be63-aa969c35a5cc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.637820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1078/2189] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:13:57.643284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.644146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.644447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.664792 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:57.665989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:57.676728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1112/2190] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:13:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:57.691043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef0bdb-7f80-4c88-be63-aa969c35a5cc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:57.699336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef0bdb-7f80-4c88-be63-aa969c35a5cc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:57.702964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330454c-dce4-47af-86b4-24e87135e144 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:57.703530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330454c-dce4-47af-86b4-24e87135e144 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.703851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330454c-dce4-47af-86b4-24e87135e144 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.704127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330454c-dce4-47af-86b4-24e87135e144 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.715202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330454c-dce4-47af-86b4-24e87135e144 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:57.716636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330454c-dce4-47af-86b4-24e87135e144 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:57.727724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1079/2191] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:57.763206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:57.764607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:57.834997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd618fce-d4a3-48c8-938c-bfc0a6b18bda None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:57.839760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:57.840397 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:57.841070 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:57.841354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] PATH_INFO: `/v3/users/4769c755b54442c493057adabcc87637` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:57.842530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4769c755b54442c493057adabcc87637)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:57.844554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:58.061306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4769c755b54442c493057adabcc87637'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:58.070832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4769c755b54442c493057adabcc87637'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:58.071564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4769c755b54442c493057adabcc87637'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:58.072295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4769c755b54442c493057adabcc87637'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:58.085449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4769c755b54442c493057adabcc87637'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:58.202866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1113/2192] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:57 2022] DELETE /identity/v3/users/4769c755b54442c493057adabcc87637 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:58.207758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:58.208055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:58.208339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:58.228557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:13:58.229682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:58.291607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1080/2193] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:13:58.302156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:58.303598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:58.318472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:13:58.368396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a6028-b6bc-4079-a370-de8c479c903e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:58.372251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:58.372865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:58.373372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:58.373896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:58.383679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:58.385114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:58.397256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1114/2194] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:13:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:13:58.895731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:58.897403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:13:58.909992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4dfaa2-6ff8-4bfe-b97c-f3cb3f8fc714 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:58.914874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:58.916294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:58.916294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:58.916294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] PATH_INFO: `/v3/projects/7e45f2de355d420bba357f8a56483314` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:58.917236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e45f2de355d420bba357f8a56483314)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:58.919204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:59.050288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e45f2de355d420bba357f8a56483314', 'request_id': 'req-26332c55-5be4-44b7-84ff-39843217fcae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.057204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e45f2de355d420bba357f8a56483314', 'request_id': 'req-26332c55-5be4-44b7-84ff-39843217fcae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.072725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e45f2de355d420bba357f8a56483314', 'request_id': 'req-26332c55-5be4-44b7-84ff-39843217fcae'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.180903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] The token cache is being invalidate because project 7e45f2de355d420bba357f8a56483314 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:13:59.180903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.181500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.183190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1081/2195] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:13:58 2022] DELETE /identity/v3/projects/7e45f2de355d420bba357f8a56483314 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:13:59.187075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:13:59.197855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3749ae9a-8318-44bf-bc01-4accac5d14e4 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:13:59.273081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71de9636-1826-4264-9660-634b34f427af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:13:59.273711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9636-1826-4264-9660-634b34f427af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:13:59.274041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9636-1826-4264-9660-634b34f427af admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:13:59.274299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71de9636-1826-4264-9660-634b34f427af admin admin] PATH_INFO: `/v3/users/2a948936ad7a4636b85299a2eca6d602` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:13:59.275418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9636-1826-4264-9660-634b34f427af admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a948936ad7a4636b85299a2eca6d602)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:13:59.277666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71de9636-1826-4264-9660-634b34f427af admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:13:59.543215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71de9636-1826-4264-9660-634b34f427af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a948936ad7a4636b85299a2eca6d602'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.551292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71de9636-1826-4264-9660-634b34f427af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a948936ad7a4636b85299a2eca6d602'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.551991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71de9636-1826-4264-9660-634b34f427af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a948936ad7a4636b85299a2eca6d602'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.552608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71de9636-1826-4264-9660-634b34f427af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a948936ad7a4636b85299a2eca6d602'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.565436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71de9636-1826-4264-9660-634b34f427af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a948936ad7a4636b85299a2eca6d602'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:13:59.660638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1115/2196] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:13:59 2022] DELETE /identity/v3/users/2a948936ad7a4636b85299a2eca6d602 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:14:00.050438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:00.052132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:00.065823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26332c55-5be4-44b7-84ff-39843217fcae admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:00.071453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:00.126378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:00.127016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.127299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.127598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] PATH_INFO: `/v3/projects/4b80629cc66c49219032ad3e16b3ca3c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.128738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b80629cc66c49219032ad3e16b3ca3c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:00.130707 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:00.250103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b80629cc66c49219032ad3e16b3ca3c', 'request_id': 'req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:00.257769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b80629cc66c49219032ad3e16b3ca3c', 'request_id': 'req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:00.264416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b80629cc66c49219032ad3e16b3ca3c', 'request_id': 'req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:00.381564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] The token cache is being invalidate because project 4b80629cc66c49219032ad3e16b3ca3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:00.382047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:00.382777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe0a6ef9-6a24-4816-a370-55f9a5e302b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:00.384356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1082/2197] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:00 2022] DELETE /identity/v3/projects/4b80629cc66c49219032ad3e16b3ca3c => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:00.389439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.390104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.390202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.415689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:00.416472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:00.457931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:00.494854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1116/2198] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:00.505111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.505416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.505666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.530598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:00.531885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:00.541048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1083/2199] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:00.545662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:00.555541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a412f-98a5-4036-9e86-2fd64dc727ca None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:00.560225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:00.560825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.561104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.561377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.562050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:14:00.562351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:00.563980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:00.576430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1117/2200] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:14:00.580895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:00.591153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577f6c8-cea1-4be3-9043-609df44d1e74 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:00.596029 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:00.596584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.596866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.597137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.597962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:00.599894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:00.715956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1084/2201] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:00 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:00.717090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:00.728465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ad335-6dce-4458-bb04-8210cd38c6b2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:00.789629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:00.790277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.790557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.790838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.791471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:00.793421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:00.920691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1118/2202] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:00 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:00.924774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:00.935764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5361571-3cef-442e-ac4f-116d08fc942c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:00.940312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:00.940910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.941191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.941468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.942145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:00.944700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:00.958511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1085/2203] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:00.959753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:00.970729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0234c4-e614-4015-9b14-196d1fb4d2de admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:00.976550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdd7e1-f197-48d7-b2c9-fdc1f520abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:00.976914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd7e1-f197-48d7-b2c9-fdc1f520abae admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:00.977196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd7e1-f197-48d7-b2c9-fdc1f520abae admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:00.977495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdd7e1-f197-48d7-b2c9-fdc1f520abae admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:00.978175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd7e1-f197-48d7-b2c9-fdc1f520abae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:00.979796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bdd7e1-f197-48d7-b2c9-fdc1f520abae admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:00.991880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1119/2204] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:14:00.996052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:01.006425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e4cfa-7273-41bf-890e-0730a5daaf29 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:01.011133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:01.011795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:01.012074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:01.012367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] PATH_INFO: `/v3/projects/29d9333d9f8646358af5d2180e774f44/users/b9fa4734769a41f5806f5c8b1438620b/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:01.028203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29d9333d9f8646358af5d2180e774f44, user_id=b9fa4734769a41f5806f5c8b1438620b, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:01.031032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:01.115508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1086/2205] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:00 2022] PUT /identity/v3/projects/29d9333d9f8646358af5d2180e774f44/users/b9fa4734769a41f5806f5c8b1438620b/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:01.125057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bee220-e8d8-46de-8e9f-d03e10f6c70d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:01.125350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bee220-e8d8-46de-8e9f-d03e10f6c70d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:01.125629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bee220-e8d8-46de-8e9f-d03e10f6c70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:01.154107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-53bee220-e8d8-46de-8e9f-d03e10f6c70d None None] MFA Rules not processed for user `b9fa4734769a41f5806f5c8b1438620b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:01.155331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-53bee220-e8d8-46de-8e9f-d03e10f6c70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:01.222515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1120/2206] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:01.234084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:01.236038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:01.280410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:01.297828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd49bf1-de14-4f85-83d3-3c2ba9ca7350 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:01.301556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:01.302823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:01.311153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:01.311153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:01.311963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:01.313461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:01.323300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1087/2207] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:03.046862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:03.047363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:03.047476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:03.066666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:03.067877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:03.093214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:03.125080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1121/2208] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:14:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:03.138464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:03.139897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:03.216353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3107b171-45fc-4a9b-a251-b7d6f79db270 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:03.221469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931f0d8-7cb9-43eb-b83b-53deaee57aee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:03.222113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931f0d8-7cb9-43eb-b83b-53deaee57aee service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:03.222399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931f0d8-7cb9-43eb-b83b-53deaee57aee service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:03.222680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931f0d8-7cb9-43eb-b83b-53deaee57aee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:03.232255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931f0d8-7cb9-43eb-b83b-53deaee57aee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:03.233874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931f0d8-7cb9-43eb-b83b-53deaee57aee service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:03.242537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1088/2209] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:14:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:14:05.713579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:05.775593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e00eb3-70c8-4eb7-b12c-2fd5f8672687 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:05.782183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:05.782841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:05.783174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:05.783453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] PATH_INFO: `/v3/users/b9fa4734769a41f5806f5c8b1438620b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:05.784624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9fa4734769a41f5806f5c8b1438620b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:05.787258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:06.415269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9fa4734769a41f5806f5c8b1438620b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:06.438199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9fa4734769a41f5806f5c8b1438620b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:06.438982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9fa4734769a41f5806f5c8b1438620b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:06.439671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9fa4734769a41f5806f5c8b1438620b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:06.460396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9fa4734769a41f5806f5c8b1438620b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:06.615907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1122/2210] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:05 2022] DELETE /identity/v3/users/b9fa4734769a41f5806f5c8b1438620b => generated 0 bytes in 915 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:06.620625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:06.621295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:06.621295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:06.643714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:06.645209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:06.747871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1089/2211] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:06.762758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:06.764337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:06.842907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab663d3-cb8b-4df3-b364-797a7d308832 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:06.847657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fb37f2-618f-4d4d-93a7-1dff51f81f01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:06.848290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb37f2-618f-4d4d-93a7-1dff51f81f01 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:06.848603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb37f2-618f-4d4d-93a7-1dff51f81f01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:06.848930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fb37f2-618f-4d4d-93a7-1dff51f81f01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:06.857572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:06.860373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb37f2-618f-4d4d-93a7-1dff51f81f01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:06.862303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fb37f2-618f-4d4d-93a7-1dff51f81f01 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:06.874139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1123/2212] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:07.319579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:07.321069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:07.335417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1382a2-c0f0-4c69-9f80-ecf0e32b6fdb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:07.340516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:07.341189 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:07.341581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:07.341919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] PATH_INFO: `/v3/projects/29d9333d9f8646358af5d2180e774f44` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:07.343246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29d9333d9f8646358af5d2180e774f44)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:07.345393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:07.511810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29d9333d9f8646358af5d2180e774f44', 'request_id': 'req-d33f8c6a-489b-4045-9df2-9dfa96d66872'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:07.518814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29d9333d9f8646358af5d2180e774f44', 'request_id': 'req-d33f8c6a-489b-4045-9df2-9dfa96d66872'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:07.535129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29d9333d9f8646358af5d2180e774f44', 'request_id': 'req-d33f8c6a-489b-4045-9df2-9dfa96d66872'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:07.537935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:07.591238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] The token cache is being invalidate because project 29d9333d9f8646358af5d2180e774f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:07.591672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:07.601916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d33f8c6a-489b-4045-9df2-9dfa96d66872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:07.604016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1090/2213] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:07 2022] DELETE /identity/v3/projects/29d9333d9f8646358af5d2180e774f44 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:07.609202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:07.609518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:07.609841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:07.638083 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:07.639314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:07.738791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1124/2214] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:07.746465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:07.747140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:07.747140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:07.792800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:07.794767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:07.805838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1091/2215] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:07.810184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:07.825577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b6908-8b66-422a-bf17-490844a2e750 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:07.840188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:07.841670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:07.841993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:07.842272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:07.842941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:14:07.843252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:07.845016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:07.861946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1125/2216] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:14:07.866416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:07.877192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e6a452-644d-4f98-9df5-ce32b9f4fe24 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:07.883960 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:07.884837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:07.885190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:07.885539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:07.886299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:07.888419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:07.995419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1092/2217] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:07 2022] POST /identity/v3/projects => generated 385 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:07.999911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:08.012001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3805b7-e15f-4b0a-bf84-fb56244ed61e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:08.087944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:08.088852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.089221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.089609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.090363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:08.092424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:08.210656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1126/2218] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:07 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:08.213446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:08.232060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32f4df-f50d-4355-82b2-9cabc630b46d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:08.240122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:08.241250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.241756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.242210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.243385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:08.246010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:08.264158 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1093/2219] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:08.265925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:08.280132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1953f59-49ae-4db4-8033-1d43c29d41f2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:08.285352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a1c60e-c7eb-4458-aa7a-24a72b793f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:08.286117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1c60e-c7eb-4458-aa7a-24a72b793f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.286478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1c60e-c7eb-4458-aa7a-24a72b793f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.286756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a1c60e-c7eb-4458-aa7a-24a72b793f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.287565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1c60e-c7eb-4458-aa7a-24a72b793f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:08.289199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a1c60e-c7eb-4458-aa7a-24a72b793f34 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:08.303690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1127/2220] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:08.308336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:08.328581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee23923a-195c-4b5f-97c1-cefa5377ecce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:08.333700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:08.334416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.334698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.334975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] PATH_INFO: `/v3/projects/23a784c5814e4c6e9bac3ac5bad27987/users/2f26f774c8ee46da8928fb33c9efed4f/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.354088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=23a784c5814e4c6e9bac3ac5bad27987, user_id=2f26f774c8ee46da8928fb33c9efed4f, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:08.356838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:08.474989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1094/2221] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:08 2022] PUT /identity/v3/projects/23a784c5814e4c6e9bac3ac5bad27987/users/2f26f774c8ee46da8928fb33c9efed4f/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:14:08.486812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.487132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.487368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.523105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] MFA Rules not processed for user `2f26f774c8ee46da8928fb33c9efed4f`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:08.524370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:08.596119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:08.617227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1128/2222] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:08.622297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:08.715548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9537a-3898-4116-929b-dcc3199b42ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:08.720839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:08.721494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.721840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.722202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.722842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:08.725558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:08.831753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1095/2223] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:08 2022] POST /identity/v3/projects => generated 385 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:14:08.832462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:08.854929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089ae2-a2a2-466c-9def-63a252c1e793 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:08.979862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:08.980786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:08.981226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:08.981676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:08.982525 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:08.984898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.076812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1129/2224] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:08 2022] POST /identity/v3/users => generated 293 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:09.081033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.092143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8df115-b6c9-4e3f-82d4-29a5a38500b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.097668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.098346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.098731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.099209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.099747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.101550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.115987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1096/2225] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:09.120007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.131267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacb6b34-0bda-4e7b-85c0-c2c154384a1d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.136171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5534b108-cf9c-4b46-b05e-f983bfd3c3f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.136962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b108-cf9c-4b46-b05e-f983bfd3c3f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.137257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b108-cf9c-4b46-b05e-f983bfd3c3f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.137516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5534b108-cf9c-4b46-b05e-f983bfd3c3f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.138210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534b108-cf9c-4b46-b05e-f983bfd3c3f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.142861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5534b108-cf9c-4b46-b05e-f983bfd3c3f2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.156305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1130/2226] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:14:09.160995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.172541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00507ccd-a134-498b-9f46-3a041ceca0a2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.178291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.179197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.179501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.179778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] PATH_INFO: `/v3/projects/cb336347d1ef4f2caaf67a1598b7ecdc/users/4a5a8ecea7674085a6a9d6e15a4701a0/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.199172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb336347d1ef4f2caaf67a1598b7ecdc, user_id=4a5a8ecea7674085a6a9d6e15a4701a0, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.201923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.295802 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1097/2227] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:09 2022] PUT /identity/v3/projects/cb336347d1ef4f2caaf67a1598b7ecdc/users/4a5a8ecea7674085a6a9d6e15a4701a0/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:09.307949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.308448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.308583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.343418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] MFA Rules not processed for user `4a5a8ecea7674085a6a9d6e15a4701a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:09.344577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:09.430170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1131/2228] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:09.434803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.499437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:09.522514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ba0e7-f3b7-4231-81cf-5a175f546c1c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.528054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.528765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.529076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.529455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.530052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.532051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.625051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1098/2229] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:09 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:09.629315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.641139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50f2a1c-4718-4aad-afb4-a1558b186dd2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.716033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.716656 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.716944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.717221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.717903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.719875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.729238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:09.882881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1132/2230] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:09 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:09.886988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.898561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0cb6d9-c863-4966-9815-4d860eabf28a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.903419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.904019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.904302 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.904582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.905222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.906881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:09.920027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1099/2231] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:09.923996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:09.935951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2ef66-7972-4207-b503-5c02c38d930b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:09.941509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:09.942542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:09.942953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:09.943315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] PATH_INFO: `/v3/projects/76957ca4f9ce4112ba719bbaf295ce99/users/c3c98f1042284605af5f1506c787f991/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:09.969289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76957ca4f9ce4112ba719bbaf295ce99, user_id=c3c98f1042284605af5f1506c787f991, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:09.972075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:10.119461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1133/2232] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:09 2022] PUT /identity/v3/projects/76957ca4f9ce4112ba719bbaf295ce99/users/c3c98f1042284605af5f1506c787f991/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:14:10.120241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:10.138884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0a6e6-49de-4a95-b67c-b5d142daf282 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:10.230584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d11b8fd-cbfb-436e-8b10-0bad23b6dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:10.231389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11b8fd-cbfb-436e-8b10-0bad23b6dffb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:10.231591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11b8fd-cbfb-436e-8b10-0bad23b6dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:10.286066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d11b8fd-cbfb-436e-8b10-0bad23b6dffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:10.286066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11b8fd-cbfb-436e-8b10-0bad23b6dffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:10.286066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d11b8fd-cbfb-436e-8b10-0bad23b6dffb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:10.286066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1100/2233] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:10 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:10.286918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:10.286918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d17d38-4a9e-4118-ad89-cdcc0e7a9699 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:10.286918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:10.286918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:10.287601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:10.287601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] PATH_INFO: `/v3/projects/76957ca4f9ce4112ba719bbaf295ce99/users/c3c98f1042284605af5f1506c787f991/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:10.287601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76957ca4f9ce4112ba719bbaf295ce99, user_id=c3c98f1042284605af5f1506c787f991, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:10.287601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:10.317985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1134/2234] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:10 2022] PUT /identity/v3/projects/76957ca4f9ce4112ba719bbaf295ce99/users/c3c98f1042284605af5f1506c787f991/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:10.341871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:10.342675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:10.342961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:10.384584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] MFA Rules not processed for user `c3c98f1042284605af5f1506c787f991`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:10.385861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:10.477814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1101/2235] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:10.494787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:10.494787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:10.561131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2518953-c4ac-468a-b5c9-145f0bf350b5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:10.565685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:10.566394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:10.566682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:10.566956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:10.640253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:10.643452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:10.656581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1135/2236] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:13.125397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:13.136973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7f9f4-f563-4859-a3f5-3c27d0a789e8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:13.142434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:13.143121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:13.143406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:13.143743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:13.188890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:13.232125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:13.235115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:13.249852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1102/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:14.866470 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:14.875835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd90177-40f1-4480-b7fb-4b17803f30b8 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:14.880230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb8472-ec6a-47c1-9506-67c9cccef4cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:14.881602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8472-ec6a-47c1-9506-67c9cccef4cb service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:14.881950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8472-ec6a-47c1-9506-67c9cccef4cb service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:14.882418 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb8472-ec6a-47c1-9506-67c9cccef4cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:14.892220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb8472-ec6a-47c1-9506-67c9cccef4cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:14.899450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb8472-ec6a-47c1-9506-67c9cccef4cb service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:14.903231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1136/2238] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:14 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:14:19.621630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:19.690509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe10ed4-897f-4dd4-ac40-b6399a8576c3 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:19.696168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:19.696844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:19.697129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:19.697409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] PATH_INFO: `/v3/users/2f26f774c8ee46da8928fb33c9efed4f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:19.698581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f26f774c8ee46da8928fb33c9efed4f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:19.700549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:19.913738 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f26f774c8ee46da8928fb33c9efed4f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:19.921121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f26f774c8ee46da8928fb33c9efed4f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:19.991776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f26f774c8ee46da8928fb33c9efed4f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:19.992412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f26f774c8ee46da8928fb33c9efed4f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.005890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f26f774c8ee46da8928fb33c9efed4f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.012385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1103/2239] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:19 2022] DELETE /identity/v3/users/2f26f774c8ee46da8928fb33c9efed4f => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:20.016938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.017390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.017531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.039106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:20.040331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:20.089970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:20.103278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1137/2240] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:20.113451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:20.114947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:20.176525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c36ca-8d5f-4ad8-a1f7-6dff0fd1c336 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:20.180629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:20.181252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.181541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.181857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.189193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:20.193212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:20.194765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:20.204799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1104/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:20.666911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:20.668359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:20.682556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893e5a6-1954-4df7-9bd9-2a5b7332a07d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:20.687502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:20.688145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.688419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.688692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] PATH_INFO: `/v3/projects/23a784c5814e4c6e9bac3ac5bad27987` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.689970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23a784c5814e4c6e9bac3ac5bad27987)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:20.691813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:20.787969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23a784c5814e4c6e9bac3ac5bad27987', 'request_id': 'req-16cb56fe-1580-4334-ac40-1a027b49eff6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.864782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23a784c5814e4c6e9bac3ac5bad27987', 'request_id': 'req-16cb56fe-1580-4334-ac40-1a027b49eff6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.872321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23a784c5814e4c6e9bac3ac5bad27987', 'request_id': 'req-16cb56fe-1580-4334-ac40-1a027b49eff6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.883877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] The token cache is being invalidate because project 23a784c5814e4c6e9bac3ac5bad27987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:20.884166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.884803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:20.886298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1138/2242] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:20 2022] DELETE /identity/v3/projects/23a784c5814e4c6e9bac3ac5bad27987 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:14:20.890468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:20.900384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8247212-441f-42f7-b3ff-4c3244c02dbf service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:20.961607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:20.962078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:20.962365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:20.962641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] PATH_INFO: `/v3/users/4a5a8ecea7674085a6a9d6e15a4701a0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:20.963785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a5a8ecea7674085a6a9d6e15a4701a0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:20.965918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:21.147850 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a5a8ecea7674085a6a9d6e15a4701a0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:21.154822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a5a8ecea7674085a6a9d6e15a4701a0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:21.195928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a5a8ecea7674085a6a9d6e15a4701a0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:21.196576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a5a8ecea7674085a6a9d6e15a4701a0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:21.209950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a5a8ecea7674085a6a9d6e15a4701a0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:21.216203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1105/2243] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:20 2022] DELETE /identity/v3/users/4a5a8ecea7674085a6a9d6e15a4701a0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:21.689125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:21.690655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:21.710775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cb56fe-1580-4334-ac40-1a027b49eff6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:21.774306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:21.775007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:21.775267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:21.775593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] PATH_INFO: `/v3/projects/cb336347d1ef4f2caaf67a1598b7ecdc` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:21.776717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb336347d1ef4f2caaf67a1598b7ecdc)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:21.778922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:21.900385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb336347d1ef4f2caaf67a1598b7ecdc', 'request_id': 'req-7d09061d-e931-4145-8822-063589e56450'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:21.995941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb336347d1ef4f2caaf67a1598b7ecdc', 'request_id': 'req-7d09061d-e931-4145-8822-063589e56450'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.002830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb336347d1ef4f2caaf67a1598b7ecdc', 'request_id': 'req-7d09061d-e931-4145-8822-063589e56450'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.014868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] The token cache is being invalidate because project cb336347d1ef4f2caaf67a1598b7ecdc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:22.015168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.015955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.017589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1139/2244] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:21 2022] DELETE /identity/v3/projects/cb336347d1ef4f2caaf67a1598b7ecdc => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:22.021601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:22.023733 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:22.036248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e2681-a76d-4b0f-aa36-f28db21644a1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:22.094469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:22.095120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:22.095402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:22.095803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] PATH_INFO: `/v3/users/c3c98f1042284605af5f1506c787f991` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:22.096970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c98f1042284605af5f1506c787f991)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:22.099086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:22.114559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:22.285270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c98f1042284605af5f1506c787f991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.292587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c98f1042284605af5f1506c787f991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.368222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c98f1042284605af5f1506c787f991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.368891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c98f1042284605af5f1506c787f991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.381508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4d76b36f-1ebc-462a-982d-798c10e748b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c98f1042284605af5f1506c787f991'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:22.388320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1106/2245] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:22 2022] DELETE /identity/v3/users/c3c98f1042284605af5f1506c787f991 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:22.822711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:22.826892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:22.841002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09061d-e931-4145-8822-063589e56450 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:22.844981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:22.899686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:22.900321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:22.900605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:22.900882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] PATH_INFO: `/v3/projects/76957ca4f9ce4112ba719bbaf295ce99` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:22.902036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] RBAC: Authorizing `identity:delete_project(project_id=76957ca4f9ce4112ba719bbaf295ce99)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:22.903974 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.019128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76957ca4f9ce4112ba719bbaf295ce99', 'request_id': 'req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:23.087488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76957ca4f9ce4112ba719bbaf295ce99', 'request_id': 'req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:23.094427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76957ca4f9ce4112ba719bbaf295ce99', 'request_id': 'req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:23.105993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] The token cache is being invalidate because project 76957ca4f9ce4112ba719bbaf295ce99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:23.106284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:23.106937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-69d0ead8-20c6-4d90-a8ec-10fc4f0bb5be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:23.108505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1140/2246] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:22 2022] DELETE /identity/v3/projects/76957ca4f9ce4112ba719bbaf295ce99 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:23.113285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.113780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.115269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.134268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:23.135488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:23.197589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1107/2247] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:23.205435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.205806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.206037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.225939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:23.227140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:23.235004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1141/2248] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:23.239353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.250692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf90f0d-12cc-4a11-b279-a5145fa66613 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.255420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.256134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.256423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.256704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.258013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:14:23.258346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.259950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.271875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1108/2249] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:14:23.276646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.286728 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21442a5f-457a-4502-96b7-ca6c1214635c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.291782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.292392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.292668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.292933 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.293598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.295777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.382136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1142/2250] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:23 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:14:23.386296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.397033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d086aca-bfd8-48ec-a272-9d76c8726000 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.474317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.475005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.475293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.475783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.476428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.478459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.581680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1109/2251] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:23 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:14:23.583030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.594689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1b0f59-65d4-4aba-9a82-0a9cb2af86ae admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.599417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.600021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.600303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.600605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.601626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.603290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.615750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1143/2252] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:23.619194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.629764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b4a5d-5ae8-49d8-9cbe-01897171c58a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.635401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d346c63-e944-481a-b190-98898ad1b8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.636017 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d346c63-e944-481a-b190-98898ad1b8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.636301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d346c63-e944-481a-b190-98898ad1b8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.636610 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d346c63-e944-481a-b190-98898ad1b8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.637256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d346c63-e944-481a-b190-98898ad1b8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.638888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d346c63-e944-481a-b190-98898ad1b8fe admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.641310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:23.651246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1110/2253] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:23.655758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.666900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1fd3b3-b9f9-4364-84e9-3290f92bc99e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.671895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.673190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.673190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.673190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] PATH_INFO: `/v3/projects/1c4cae65522c475fad31e57ae71c3c5f/users/9e644ca2991040f8bb1c70599db0cb0f/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.688750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c4cae65522c475fad31e57ae71c3c5f, user_id=9e644ca2991040f8bb1c70599db0cb0f, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.691500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.769229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1144/2254] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:23 2022] PUT /identity/v3/projects/1c4cae65522c475fad31e57ae71c3c5f/users/9e644ca2991040f8bb1c70599db0cb0f/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:23.780618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1c33-8977-4682-93e6-2f8de2e49089 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.780905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1c33-8977-4682-93e6-2f8de2e49089 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.781183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973d1c33-8977-4682-93e6-2f8de2e49089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.811401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-973d1c33-8977-4682-93e6-2f8de2e49089 None None] MFA Rules not processed for user `9e644ca2991040f8bb1c70599db0cb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:23.812630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-973d1c33-8977-4682-93e6-2f8de2e49089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:23.887688 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1111/2255] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:23.898515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:23.900168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:23.938220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:23.960524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b172864-f16b-4c39-a76e-f1721f3a627c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:23.964260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1292af-c586-4c99-81d4-fb502d6fe060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:23.965159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1292af-c586-4c99-81d4-fb502d6fe060 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:23.965159 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1292af-c586-4c99-81d4-fb502d6fe060 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:23.965444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1292af-c586-4c99-81d4-fb502d6fe060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:23.973949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1292af-c586-4c99-81d4-fb502d6fe060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:23.975400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1292af-c586-4c99-81d4-fb502d6fe060 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:23.985114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1145/2256] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:25.750800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:25.751324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:25.751438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:25.779980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] MFA Rules not processed for user `9e644ca2991040f8bb1c70599db0cb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:25.781423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:25.791951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1112/2257] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:28.982037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:28.982631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:28.982631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:29.003150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:29.004351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:29.067013 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1146/2258] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 30 01:14:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:29.084303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:29.086611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:29.164698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a464e83-8ebe-4acc-9fb1-ab42bdc51276 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:29.170155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce620086-3f3e-4b6a-807c-f086bab89af5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:29.170910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce620086-3f3e-4b6a-807c-f086bab89af5 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:29.171055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce620086-3f3e-4b6a-807c-f086bab89af5 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:29.171657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce620086-3f3e-4b6a-807c-f086bab89af5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:29.185349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce620086-3f3e-4b6a-807c-f086bab89af5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:29.186963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce620086-3f3e-4b6a-807c-f086bab89af5 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:29.196842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1113/2259] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:14:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:14:33.203006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:33.204446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:33.271258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206ce4f-4767-4e0c-b115-398bc1bbd00f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:33.276498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:33.277154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:33.277456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:33.277747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] PATH_INFO: `/v3/users/aaf5709310884616acc4beebbdb3cf4b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:33.278900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf5709310884616acc4beebbdb3cf4b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:33.280811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:33.473513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf5709310884616acc4beebbdb3cf4b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:33.481270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf5709310884616acc4beebbdb3cf4b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:33.481995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf5709310884616acc4beebbdb3cf4b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:33.482626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf5709310884616acc4beebbdb3cf4b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:33.496388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf5709310884616acc4beebbdb3cf4b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:33.575130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:33.579711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1147/2260] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:33 2022] DELETE /identity/v3/users/aaf5709310884616acc4beebbdb3cf4b => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:33.584448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:33.585042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:33.585042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:33.604313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:33.605640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:33.667434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1114/2261] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:33.677759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:33.679887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:33.739246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729fae05-c955-4267-880e-d5594899bd36 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:33.743011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747a52e8-74e6-4c18-8c45-d2827d66aa60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:33.743628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a52e8-74e6-4c18-8c45-d2827d66aa60 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:33.743907 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a52e8-74e6-4c18-8c45-d2827d66aa60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:33.744163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747a52e8-74e6-4c18-8c45-d2827d66aa60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:33.753284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a52e8-74e6-4c18-8c45-d2827d66aa60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:33.754828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747a52e8-74e6-4c18-8c45-d2827d66aa60 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:33.764586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1148/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:34.157800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.159595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:34.180811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd10d9-f47f-4422-b494-227ee034a409 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.186994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:34.187613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.187901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.188377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] PATH_INFO: `/v3/projects/273553011da245b5b18e2c606124ed1a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.189763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=273553011da245b5b18e2c606124ed1a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:34.192116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:34.200495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:34.294335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273553011da245b5b18e2c606124ed1a', 'request_id': 'req-42659aa9-6417-496d-8838-0c185dac13c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:34.300816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273553011da245b5b18e2c606124ed1a', 'request_id': 'req-42659aa9-6417-496d-8838-0c185dac13c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:34.307618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273553011da245b5b18e2c606124ed1a', 'request_id': 'req-42659aa9-6417-496d-8838-0c185dac13c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:34.393210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] The token cache is being invalidate because project 273553011da245b5b18e2c606124ed1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:34.393551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:34.394216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-42659aa9-6417-496d-8838-0c185dac13c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:34.396198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1115/2263] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:34 2022] DELETE /identity/v3/projects/273553011da245b5b18e2c606124ed1a => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:34.401607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.402122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.402317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.424426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:34.425659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:34.511550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1149/2264] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:34.519379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.519664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.519937 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.549329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:34.549329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:34.557697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1116/2265] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:34.562212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.575650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4490741-e4cc-4246-9cf2-ed80e722606e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.581449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:34.582687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.582687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.582687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.583310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:14:34.583618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:34.585858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:34.599930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1150/2266] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:14:34.604793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.616710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa13394-1c5c-4d54-a2da-ce1343aff642 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.625144 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:34.626874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.626874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.626874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.627243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:34.629439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:34.735086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1117/2267] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:34 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:34.739399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.749639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7c6f3-1aa0-4809-a7ed-b908b985673c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.810720 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:34.811251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.811535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.811812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.812443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:34.814417 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:34.939222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1151/2268] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:34 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:34.943965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.958398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c619c9-5c22-4727-b4f4-70182e8308e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.963830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:34.964431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:34.965494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:34.965494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:34.965706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:34.971484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:34.979165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1118/2269] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:34.983665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:34.994832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d9e00-ac1d-4e4a-b9db-45e2f885fbdd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:34.999515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cd996-a70b-4cee-8ffd-9f946d6a1044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:35.000113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cd996-a70b-4cee-8ffd-9f946d6a1044 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:35.000396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cd996-a70b-4cee-8ffd-9f946d6a1044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:35.000672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cd996-a70b-4cee-8ffd-9f946d6a1044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:35.001312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cd996-a70b-4cee-8ffd-9f946d6a1044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:35.002968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cd996-a70b-4cee-8ffd-9f946d6a1044 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:35.013604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1152/2270] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:35.017847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:35.028211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0928009f-5cee-4865-9909-f4de61035bab admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:35.032695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:35.033367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:35.033648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:35.033954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] PATH_INFO: `/v3/projects/89afb17ea6c04c3b9f78d66a87dfddcc/users/b8790a3d16ad45da8c457af3e0d5aa31/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:35.049470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89afb17ea6c04c3b9f78d66a87dfddcc, user_id=b8790a3d16ad45da8c457af3e0d5aa31, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:35.052174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:35.122097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1119/2271] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:35 2022] PUT /identity/v3/projects/89afb17ea6c04c3b9f78d66a87dfddcc/users/b8790a3d16ad45da8c457af3e0d5aa31/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:35.141178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:35.141473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:35.141851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:35.161865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:35.171593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] MFA Rules not processed for user `b8790a3d16ad45da8c457af3e0d5aa31`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:35.172810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:35.237544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1153/2272] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:35.248786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:35.250477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:35.308137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff697fc6-beb2-434e-8a6d-ec454a4c58c3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:35.311810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e34e950-7ae4-43e4-9a8e-f488f7b95a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:35.312443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e950-7ae4-43e4-9a8e-f488f7b95a8e service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:35.312724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e950-7ae4-43e4-9a8e-f488f7b95a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:35.312997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e34e950-7ae4-43e4-9a8e-f488f7b95a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:35.321995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e950-7ae4-43e4-9a8e-f488f7b95a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:35.323420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e34e950-7ae4-43e4-9a8e-f488f7b95a8e service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:35.332124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1120/2273] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:35.359824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:35.429888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62dcdd9-a191-4204-a7b2-4e1891743eae None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:35.434482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:35.435114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:35.445710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:35.445710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] PATH_INFO: `/v3/users/b8790a3d16ad45da8c457af3e0d5aa31` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:35.445710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8790a3d16ad45da8c457af3e0d5aa31)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:35.445710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:35.758835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8790a3d16ad45da8c457af3e0d5aa31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:35.773274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8790a3d16ad45da8c457af3e0d5aa31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:35.774712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8790a3d16ad45da8c457af3e0d5aa31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:35.775594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8790a3d16ad45da8c457af3e0d5aa31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:35.800286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8790a3d16ad45da8c457af3e0d5aa31'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:35.876545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1154/2274] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:35 2022] DELETE /identity/v3/users/b8790a3d16ad45da8c457af3e0d5aa31 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:35.883722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:35.884247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:35.884816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:35.918373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:35.919916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:35.950804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:35.983575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1121/2275] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:14:35.993347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:35.994849 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:36.014991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:36.055716 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe6361a-e1a3-4337-8be7-7cb619079bd1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:36.059617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06efd01-352e-43a8-8090-0448daf09e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:36.060237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06efd01-352e-43a8-8090-0448daf09e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:36.060523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06efd01-352e-43a8-8090-0448daf09e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:36.060801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06efd01-352e-43a8-8090-0448daf09e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:36.073682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06efd01-352e-43a8-8090-0448daf09e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:36.075264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06efd01-352e-43a8-8090-0448daf09e52 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:36.086522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1155/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:36.530047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:36.531762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:36.545015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9c7a-88a8-4e22-bfb8-18685f2410e7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:36.550682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:36.551387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:36.551750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:36.552094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] PATH_INFO: `/v3/projects/89afb17ea6c04c3b9f78d66a87dfddcc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:36.553781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89afb17ea6c04c3b9f78d66a87dfddcc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:36.555952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:36.689964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89afb17ea6c04c3b9f78d66a87dfddcc', 'request_id': 'req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:36.697226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89afb17ea6c04c3b9f78d66a87dfddcc', 'request_id': 'req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:36.703576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89afb17ea6c04c3b9f78d66a87dfddcc', 'request_id': 'req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:36.795653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] The token cache is being invalidate because project 89afb17ea6c04c3b9f78d66a87dfddcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:14:36.795948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:36.798916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2dd11abb-b438-47ad-a4b0-5d5bd5a38f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:36.798916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1122/2277] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:14:36 2022] DELETE /identity/v3/projects/89afb17ea6c04c3b9f78d66a87dfddcc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:36.803050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:36.804046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:36.804477 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:36.825749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:36.826998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:36.908590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1156/2278] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:36.919501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:36.920106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:36.920106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:36.952718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:36.954559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:36.977297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1123/2279] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:36.982783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.001922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81f306-6717-4b36-b3b2-dbb33856bc9f None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.008043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.008687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.009012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.009264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.010079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:14:37.010503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.013608 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.036554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1157/2280] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:14:37.043121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.057951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5780e0-8f9d-4a2b-bf7b-75509d4fb9a9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.068669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.069708 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.070052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.070359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.071061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.073608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.181422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1124/2281] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:14:37 2022] POST /identity/v3/projects => generated 391 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:37.186456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.200755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91422614-88b8-47fa-ad75-6b6f547c4fdc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.286078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.286699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.286981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.287252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.288109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.291318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.402972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1158/2282] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:14:37 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:14:37.409536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.423676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6fd6f-8960-4048-a83f-f3ac23aad9b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.429433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.430233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.430515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.430791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.431439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.433143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.445344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1125/2283] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:37.449523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.461577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0dbce-f927-4c06-b8c5-eda301157525 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.467297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8301f9-c052-4914-9de4-18f317df97ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.467929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8301f9-c052-4914-9de4-18f317df97ab admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.468215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8301f9-c052-4914-9de4-18f317df97ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.468540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8301f9-c052-4914-9de4-18f317df97ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.469234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8301f9-c052-4914-9de4-18f317df97ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.471580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8301f9-c052-4914-9de4-18f317df97ab admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.487780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1159/2284] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:14:37.493374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.513126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bba92b-7a16-4b26-b4c3-008b12e46945 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.520497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.521135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.521414 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.521694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] PATH_INFO: `/v3/projects/06a0d78fbb82455981e28463b9f64d16/users/0e4fd08dba054affa9a70d98de5cad0d/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.538921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=06a0d78fbb82455981e28463b9f64d16, user_id=0e4fd08dba054affa9a70d98de5cad0d, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.542859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.603941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1126/2285] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:14:37 2022] PUT /identity/v3/projects/06a0d78fbb82455981e28463b9f64d16/users/0e4fd08dba054affa9a70d98de5cad0d/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:14:37.616587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981b154-6959-4cf3-b4a9-ed082df2277b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.616898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981b154-6959-4cf3-b4a9-ed082df2277b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.617247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1981b154-6959-4cf3-b4a9-ed082df2277b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.654275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1981b154-6959-4cf3-b4a9-ed082df2277b None None] MFA Rules not processed for user `0e4fd08dba054affa9a70d98de5cad0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:37.656496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1981b154-6959-4cf3-b4a9-ed082df2277b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:37.675717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:37.736724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1160/2286] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:37.748081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:37.749947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:37.809425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbd007-3b13-4e70-bac2-8d9e6cb0e41a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:37.813106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:37.813713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:37.814031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:37.814303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:37.821329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:37.823780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:37.825321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:37.835683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1127/2287] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:14:47.733723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=20 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:48.012863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:55.786897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42caff-7267-436e-b2aa-c48baed82777 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:55.787390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42caff-7267-436e-b2aa-c48baed82777 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:55.787509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c42caff-7267-436e-b2aa-c48baed82777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:55.810433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7c42caff-7267-436e-b2aa-c48baed82777 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:55.811713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7c42caff-7267-436e-b2aa-c48baed82777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:55.892683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1161/2288] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:14:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:55.906678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:55.908510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:56.032860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511c445a-cddd-4402-bfec-23d06de74fea service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:56.041899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:56.042565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:56.043000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:56.043539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:56.053713 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:56.055949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:56.065919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1128/2289] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:14:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:14:57.062249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0c243-1869-4d34-a837-b5358aed09d1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:57.062924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0c243-1869-4d34-a837-b5358aed09d1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:57.066830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0c243-1869-4d34-a837-b5358aed09d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:57.084224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-30a0c243-1869-4d34-a837-b5358aed09d1 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:57.085514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-30a0c243-1869-4d34-a837-b5358aed09d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:57.093620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1162/2290] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 30 01:14:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:57.107673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:57.118255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c1c0e-ffba-47c4-9c0c-3a013c4bb313 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:57.123501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:57.126862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:57.126862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:57.126862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:57.134889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:57.136585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:57.149165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1129/2291] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:14:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:14:57.857906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52739fc-ad58-49cb-bc88-ccf4285608ad None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:57.858253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52739fc-ad58-49cb-bc88-ccf4285608ad None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:57.858456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52739fc-ad58-49cb-bc88-ccf4285608ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:57.879185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a52739fc-ad58-49cb-bc88-ccf4285608ad None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:57.880387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a52739fc-ad58-49cb-bc88-ccf4285608ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:57.888365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1163/2292] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:14:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:57.902234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:57.913793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a485e3-a5f6-494e-8a46-1821e937d1c8 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:57.922463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:57.923460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:57.923747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:57.924024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:57.933815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:57.935565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:57.944726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1130/2293] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:14:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:14:58.542400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:58.542875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:58.542962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:58.564049 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:58.565278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:58.574215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1164/2294] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 30 01:14:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:58.588253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:58.599458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8018b5-b8b6-42a2-800b-6ed0fdd37436 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:58.605034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c88886-05c8-4ed1-941b-55cb655d7693 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:58.610034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c88886-05c8-4ed1-941b-55cb655d7693 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:58.610034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c88886-05c8-4ed1-941b-55cb655d7693 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:58.610034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c88886-05c8-4ed1-941b-55cb655d7693 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:58.615452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c88886-05c8-4ed1-941b-55cb655d7693 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:58.617150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c88886-05c8-4ed1-941b-55cb655d7693 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:58.625905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1131/2295] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:14:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:14:59.241268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:59.308725 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d75c54-e360-4d57-8858-e400b6ab25c5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:59.313531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:59.314209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:59.314494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:59.314951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] PATH_INFO: `/v3/users/0e4fd08dba054affa9a70d98de5cad0d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:59.315886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e4fd08dba054affa9a70d98de5cad0d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:59.317908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:59.479045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e4fd08dba054affa9a70d98de5cad0d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:59.486870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e4fd08dba054affa9a70d98de5cad0d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:59.487882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e4fd08dba054affa9a70d98de5cad0d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:59.488517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e4fd08dba054affa9a70d98de5cad0d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:59.504010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e4fd08dba054affa9a70d98de5cad0d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:14:59.554953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1165/2296] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:14:59 2022] DELETE /identity/v3/users/0e4fd08dba054affa9a70d98de5cad0d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:14:59.560162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:59.560496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:59.560736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:59.585579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:14:59.587476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:59.649977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1132/2297] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:14:59.660681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:14:59.662135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:14:59.669412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:14:59.731312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a60cc8-1e6b-4719-9365-d3daf8f3b8db admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:14:59.735203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:14:59.735845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:14:59.736127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:14:59.736402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:14:59.747500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:14:59.748984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:14:59.760942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1166/2298] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:00.209334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:00.213645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:00.224780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0a47ec-0f7d-4caa-9e03-fb29dccb0d45 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:00.230125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:00.231024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:00.231497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:00.232385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] PATH_INFO: `/v3/projects/06a0d78fbb82455981e28463b9f64d16` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:00.233804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=06a0d78fbb82455981e28463b9f64d16)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:00.235912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:00.384504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:00.387585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06a0d78fbb82455981e28463b9f64d16', 'request_id': 'req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:00.394072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06a0d78fbb82455981e28463b9f64d16', 'request_id': 'req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:00.400555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06a0d78fbb82455981e28463b9f64d16', 'request_id': 'req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:00.486050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] The token cache is being invalidate because project 06a0d78fbb82455981e28463b9f64d16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:15:00.486298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:00.486913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65a05946-1dd6-4df9-95a5-f22dbefa7c1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:00.488432 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1133/2299] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:15:00 2022] DELETE /identity/v3/projects/06a0d78fbb82455981e28463b9f64d16 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:01.361017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:01.362463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:01.375503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b5536-9e81-4b7a-a91b-f16187962bf5 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:01.435431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:01.436112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:01.436395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:01.436673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] PATH_INFO: `/v3/users/9e644ca2991040f8bb1c70599db0cb0f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:01.437844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e644ca2991040f8bb1c70599db0cb0f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:01.439826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:01.615912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e644ca2991040f8bb1c70599db0cb0f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:01.624571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e644ca2991040f8bb1c70599db0cb0f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:01.625301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e644ca2991040f8bb1c70599db0cb0f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:01.625945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e644ca2991040f8bb1c70599db0cb0f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:01.640448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e644ca2991040f8bb1c70599db0cb0f'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:01.709405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1167/2300] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:15:01 2022] DELETE /identity/v3/users/9e644ca2991040f8bb1c70599db0cb0f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:01.714528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:01.715025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:01.715122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:01.734293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:01.735505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:01.795633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1134/2301] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:01.805432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:01.806952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:01.838665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:01.865276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970a0ca-550f-4ffb-b9df-93f0e6f674e8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:01.869584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f3873-d6af-463c-a469-6606211ab192 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:01.870402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f3873-d6af-463c-a469-6606211ab192 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:01.871068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f3873-d6af-463c-a469-6606211ab192 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:01.871444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f3873-d6af-463c-a469-6606211ab192 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:01.880965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f3873-d6af-463c-a469-6606211ab192 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:01.882537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f3873-d6af-463c-a469-6606211ab192 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:01.893134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1168/2302] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:02.141871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.144716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:02.156806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a3c6f-6d38-44cd-88de-c6e485f094e8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.161552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.162649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.162649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.167785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] PATH_INFO: `/v3/projects/1c4cae65522c475fad31e57ae71c3c5f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.167785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c4cae65522c475fad31e57ae71c3c5f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.167785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.265534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c4cae65522c475fad31e57ae71c3c5f', 'request_id': 'req-77154795-d77d-4d50-9b46-220877437c91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:02.271962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c4cae65522c475fad31e57ae71c3c5f', 'request_id': 'req-77154795-d77d-4d50-9b46-220877437c91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:02.277774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c4cae65522c475fad31e57ae71c3c5f', 'request_id': 'req-77154795-d77d-4d50-9b46-220877437c91'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:02.355812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] The token cache is being invalidate because project 1c4cae65522c475fad31e57ae71c3c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:15:02.356264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:02.356994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-77154795-d77d-4d50-9b46-220877437c91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:15:02.358693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1135/2303] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:15:02 2022] DELETE /identity/v3/projects/1c4cae65522c475fad31e57ae71c3c5f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:02.363718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.364217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.364352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.384866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:02.386078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:02.455506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1169/2304] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:02.461972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.462278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.462557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.485490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:02.485711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:02.494794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1136/2305] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:02.499099 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.511760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc81832-d890-445a-b4b2-cbebb4dcc022 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.516675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.517280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.517561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.517870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.518611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:15:02.518944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.520554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.536920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1170/2306] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:15:02.538258 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.549261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7630140-5cff-4ed4-8034-ad34c908f702 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.554193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.554892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.555236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.555569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.556951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.558870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.669199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1137/2307] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:15:02 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:02.670479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.683485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f51fb2-6aff-4903-bca4-b2baceeac71d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.744690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.745304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.745587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.745897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.746584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.748674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.826947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1171/2308] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:15:02 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:02.828182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.839596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018019a-447b-49b0-b01a-ea7ffae1c15e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.844514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.845167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.845450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.845760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.846412 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.848016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.859978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1138/2309] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:02.864104 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.874811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65643a52-4f0d-4a9e-9e69-e26a6de721bb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.879536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8badb37-3721-4717-a11d-5486b191cf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.880137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8badb37-3721-4717-a11d-5486b191cf31 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.880412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8badb37-3721-4717-a11d-5486b191cf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.880684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8badb37-3721-4717-a11d-5486b191cf31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.881326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8badb37-3721-4717-a11d-5486b191cf31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.883625 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8badb37-3721-4717-a11d-5486b191cf31 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.894873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1172/2310] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:02.898989 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:02.909800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0d62f-e0e5-454c-b322-8f62fdf69b25 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:02.914515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:02.915796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:02.916175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:02.916547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] PATH_INFO: `/v3/projects/c6f3997439524d4583ece79273a55da4/users/fdf94212f7654ff396be6dc2c2ca4874/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:02.933226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f3997439524d4583ece79273a55da4, user_id=fdf94212f7654ff396be6dc2c2ca4874, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:02.935921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:02.938235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:03.053982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1139/2311] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:15:02 2022] PUT /identity/v3/projects/c6f3997439524d4583ece79273a55da4/users/fdf94212f7654ff396be6dc2c2ca4874/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:15:03.059898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6eae4-bcb7-43ae-addd-b6d3e4d45ca8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:03.060203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6eae4-bcb7-43ae-addd-b6d3e4d45ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:03.060479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6eae4-bcb7-43ae-addd-b6d3e4d45ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:03.085775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a3b6eae4-bcb7-43ae-addd-b6d3e4d45ca8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:03.086988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a3b6eae4-bcb7-43ae-addd-b6d3e4d45ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:03.131956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:03.152979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1173/2312] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:03.163012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:03.168551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:03.234953 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744440f1-b51b-4173-bb66-a950fa55fc81 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:03.241010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:03.242106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:03.242606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:03.243140 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:03.257778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:03.260306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:03.268934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1140/2313] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:09.424256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:09.425185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:09.425528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:09.458785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] MFA Rules not processed for user `fdf94212f7654ff396be6dc2c2ca4874`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:09.461163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:09.542785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:09.550067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1174/2314] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:09.560224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2a771-0664-4d15-a63f-c343bcc3ca16 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:09.567941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:09.568967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:09.577770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:09.577770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:09.577770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:09.577770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:09.680584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1141/2315] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:15:09 2022] POST /identity/v3/projects => generated 383 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:09.682246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:09.695962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965db0-18d1-4423-80fa-e93ec35bb43a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:09.767196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:09.767914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:09.768201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:09.768476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:09.769115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:09.771094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:09.899271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1175/2316] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:15:09 2022] POST /identity/v3/users => generated 298 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:15:09.905504 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:09.917861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91133479-5930-4aa4-bc3c-e3a7fc4154cd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:09.925341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:09.925341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:09.925341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:09.925341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:09.926094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:09.926811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:09.942165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1142/2317] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:09.943816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:09.955235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d5731-03bf-4dbe-abef-0e8709a3f5fc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:09.960084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:09.960749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:09.961032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:09.961312 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] PATH_INFO: `/v3/projects/66adbbaaffa64f48aa2dc9417803c9bb/users/87e615b6b71f46d0b4d6c1867b6b58a2/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:09.983106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66adbbaaffa64f48aa2dc9417803c9bb, user_id=87e615b6b71f46d0b4d6c1867b6b58a2, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:09.986201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:10.075684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1176/2318] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:15:09 2022] PUT /identity/v3/projects/66adbbaaffa64f48aa2dc9417803c9bb/users/87e615b6b71f46d0b4d6c1867b6b58a2/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:10.079900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:10.091725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ad4f7d-e7ac-42e8-ad8f-f9b71f01866f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:10.173494 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7708f7-1c53-4e43-9a88-a8a1ffdcd387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:10.174254 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7708f7-1c53-4e43-9a88-a8a1ffdcd387 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:10.174592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7708f7-1c53-4e43-9a88-a8a1ffdcd387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:10.174881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7708f7-1c53-4e43-9a88-a8a1ffdcd387 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:10.175536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7708f7-1c53-4e43-9a88-a8a1ffdcd387 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:10.177376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7708f7-1c53-4e43-9a88-a8a1ffdcd387 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:10.190669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1143/2319] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:15:10.194091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:10.206103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be6d8ce-55ac-4c5b-bc1b-864fda53ea95 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:10.211647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d0dd1-3c46-4aef-8e3a-139b3e89eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:10.214557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d0dd1-3c46-4aef-8e3a-139b3e89eda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:10.214839 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d0dd1-3c46-4aef-8e3a-139b3e89eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:10.215165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d0dd1-3c46-4aef-8e3a-139b3e89eda1 admin admin] PATH_INFO: `/v3/projects/66adbbaaffa64f48aa2dc9417803c9bb/users/87e615b6b71f46d0b4d6c1867b6b58a2/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:10.217121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d0dd1-3c46-4aef-8e3a-139b3e89eda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66adbbaaffa64f48aa2dc9417803c9bb, user_id=87e615b6b71f46d0b4d6c1867b6b58a2, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:10.220179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d0dd1-3c46-4aef-8e3a-139b3e89eda1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:10.234086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1177/2320] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:15:10 2022] PUT /identity/v3/projects/66adbbaaffa64f48aa2dc9417803c9bb/users/87e615b6b71f46d0b4d6c1867b6b58a2/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:15:13.102558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:13.103311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:13.103311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:13.138177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] MFA Rules not processed for user `942d175e7e2742728e66fea84c851038`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:13.140815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:13.278131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1144/2321] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:17.140288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ad267-9ddd-400d-9c0b-02dd2fb4ae93 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:17.140755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ad267-9ddd-400d-9c0b-02dd2fb4ae93 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:17.140867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ad267-9ddd-400d-9c0b-02dd2fb4ae93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:17.178227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ab3ad267-9ddd-400d-9c0b-02dd2fb4ae93 None None] MFA Rules not processed for user `87e615b6b71f46d0b4d6c1867b6b58a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:17.179516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ab3ad267-9ddd-400d-9c0b-02dd2fb4ae93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:17.186161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:17.262611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1178/2322] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:17.280249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:17.282296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:17.289214 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:17.359575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62323fb6-dbfa-4237-9b3c-d984f58e552e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:17.363371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:17.363996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:17.364277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:17.364554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:17.459480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:17.461015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:17.481229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1145/2323] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:17.853977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc140-7be2-4e43-883f-ada3a3afaacd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:17.858542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc140-7be2-4e43-883f-ada3a3afaacd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:17.858542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165fc140-7be2-4e43-883f-ada3a3afaacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:17.877345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-165fc140-7be2-4e43-883f-ada3a3afaacd None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:17.878989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-165fc140-7be2-4e43-883f-ada3a3afaacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:17.971252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1179/2324] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:15:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:17.985702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:17.987386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:18.085681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c876c-3e7b-4e40-a133-03675d43cc1f service glance] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:18.092001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02d105-2ee0-4191-a04d-7c7f7d21bd1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:18.092585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02d105-2ee0-4191-a04d-7c7f7d21bd1a service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:18.092861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02d105-2ee0-4191-a04d-7c7f7d21bd1a service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:18.093269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02d105-2ee0-4191-a04d-7c7f7d21bd1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:18.102032 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02d105-2ee0-4191-a04d-7c7f7d21bd1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:18.108363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02d105-2ee0-4191-a04d-7c7f7d21bd1a service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:18.113830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1146/2325] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:15:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:15:24.725657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651200-7b4e-4d5e-a49d-bf1727a91269 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:24.726638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651200-7b4e-4d5e-a49d-bf1727a91269 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:24.727098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a651200-7b4e-4d5e-a49d-bf1727a91269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:24.751196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7a651200-7b4e-4d5e-a49d-bf1727a91269 None None] MFA Rules not processed for user `76ea3ed3bb2b412093004c7e0f49c7ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:24.752578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7a651200-7b4e-4d5e-a49d-bf1727a91269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:24.832294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1180/2326] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:28.109608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=78 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:30.534992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f221-22a8-4995-ba4b-934caea8c95f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:30.535681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f221-22a8-4995-ba4b-934caea8c95f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:30.535681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837f221-22a8-4995-ba4b-934caea8c95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:30.568435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8837f221-22a8-4995-ba4b-934caea8c95f None None] MFA Rules not processed for user `2ffe5c7cb44f476e913d64003105002c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:30.569855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8837f221-22a8-4995-ba4b-934caea8c95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:30.644092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1147/2327] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:30.877192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:30.878040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:30.878335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:30.904143 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] MFA Rules not processed for user `fdf94212f7654ff396be6dc2c2ca4874`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:30.905398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:30.915064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1181/2328] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:37.257245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:37.257739 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:37.257879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:37.278042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:37.279322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:37.286766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1148/2329] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:15:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:37.299281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:37.311934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16a683-9b2a-48f4-ae52-8cd32a13dcd1 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:37.317461 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:37.318117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:37.318400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:37.318776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:37.328744 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:37.330598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:37.339307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1182/2330] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:15:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:15:41.940562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:42.036576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f084e61b-73a9-47de-8a9c-9e4caedf352f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:42.043072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:42.043934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:42.044301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:42.044843 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:42.046973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:42.144849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:42.146478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b454b-763d-4db7-b1e4-1e18ade397c4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:42.166223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1149/2331] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:42.668783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:42.670983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:42.684497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f9862-8109-4488-89c5-e113bea37120 service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:42.688685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:42.689483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:42.689832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:42.690282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:42.692058 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:42.707647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:42.709536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:42.736210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1183/2332] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:42.850104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:42.850523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:42.850907 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:42.874099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] MFA Rules not processed for user `90486e175bb2411b9f2adedf6e3ecac6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:15:42.877013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:42.917961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:42.943340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1150/2333] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:15:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:15:51.950752 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:51.952156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:51.960419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:51.967171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09333afc-8c90-4c90-b43d-63b95983c969 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:51.972818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:51.980506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:51.980506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:51.980506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:51.980506 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:51.986764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:51.988377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:51.998458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1184/2334] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:15:53.012484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:15:59.391107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:15:59.393626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:59.410918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1538ac-e573-4cd1-963a-43f4f1d4b003 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:15:59.418448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:15:59.419099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:15:59.422212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:15:59.422212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:15:59.422212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:15:59.433469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:15:59.435313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:15:59.447347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1151/2335] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:15:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:02.000127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:09.440801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:11.363694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:11.373216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27df106e-66be-4742-9691-c3e4c50b0996 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:11.376837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:11.377431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:11.377714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:11.378024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:11.388195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:11.393262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:11.400276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1185/2336] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:12.070815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:12.072874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:12.132402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00796e86-8ad5-46a7-9da0-238e69867059 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:12.136483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:12.137101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:12.137381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:12.137657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:12.139452 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:12.150752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:12.152192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:12.161519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1152/2337] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:12 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:16:13.512949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:13.522009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d13da4-496d-4c8f-8b0d-e89efa738927 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:13.526327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:13.526929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:13.527210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:13.527513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:13.529251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:13.539305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:13.542111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:13.551253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1186/2338] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:21.076246 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:21.089838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578763cc-7002-486f-b700-bd25a56c4579 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:21.095534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:21.096155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:21.096417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:21.096689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:21.098498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:21.112765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:21.114538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:21.127482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1153/2339] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:23.547115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=12 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:23.626120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:23.639742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe17feb-2c3e-406a-bb36-ab9c0d8456c0 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:23.645403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:23.646061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:23.646343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:23.646620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:23.648514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:23.660572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:23.662280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:23.673223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1187/2340] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:31.123183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=32 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:33.668628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:39.660958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:39.671355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa39e9-dfbb-42ba-aeb9-13e2a1c07f48 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:39.675082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:39.678928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:39.678928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:39.678928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:39.678928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:39.745780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:39.747222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:39.759374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1154/2341] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:43.606661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:43.613190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:43.631488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0ad2e-43a6-4f17-b771-873979589c6d service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:43.637274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:43.638023 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:43.638363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:43.639724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] PATH_INFO: `/v3/users/76ea3ed3bb2b412093004c7e0f49c7ce` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:43.641084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76ea3ed3bb2b412093004c7e0f49c7ce)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:43.644060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:43.865872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76ea3ed3bb2b412093004c7e0f49c7ce'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.873679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76ea3ed3bb2b412093004c7e0f49c7ce'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.874711 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76ea3ed3bb2b412093004c7e0f49c7ce'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.875326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76ea3ed3bb2b412093004c7e0f49c7ce'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.887371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-245fec88-a16a-499e-ac51-18705805cff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76ea3ed3bb2b412093004c7e0f49c7ce'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:43.964982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1188/2342] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:16:43 2022] DELETE /identity/v3/users/76ea3ed3bb2b412093004c7e0f49c7ce => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:44.217616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.239027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a3a32-52a6-4f18-bd95-e1438150546a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.303164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.303808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.304104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.304377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] PATH_INFO: `/v3/projects/9f0605a8a2fe48e49a410127b5f3b32c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.305515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0605a8a2fe48e49a410127b5f3b32c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.307466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.414899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0605a8a2fe48e49a410127b5f3b32c', 'request_id': 'req-a6c0e48a-5788-419b-a55a-7bcef65d8416'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:44.424485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0605a8a2fe48e49a410127b5f3b32c', 'request_id': 'req-a6c0e48a-5788-419b-a55a-7bcef65d8416'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:44.431231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0605a8a2fe48e49a410127b5f3b32c', 'request_id': 'req-a6c0e48a-5788-419b-a55a-7bcef65d8416'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:44.536609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] The token cache is being invalidate because project 9f0605a8a2fe48e49a410127b5f3b32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:16:44.536902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:44.537536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a6c0e48a-5788-419b-a55a-7bcef65d8416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:16:44.539084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1155/2343] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:16:44 2022] DELETE /identity/v3/projects/9f0605a8a2fe48e49a410127b5f3b32c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:44.544409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.545512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.545923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.566439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:44.567912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.636643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1189/2344] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:44.641625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.641950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.642275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.664898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:44.666449 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:44.675527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1156/2345] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:44.680103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.690032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924285dc-caae-4491-b9ca-d1369a9b6ace None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.695287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.696041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.696433 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.696833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.697787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:16:44.698378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.700227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.715502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1190/2346] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:16:44.720591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.730220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f4a59-00cd-4287-a564-652fcfe48f2a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.738053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.738689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.738977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.739250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.739900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.741968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.765637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:44.861023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1157/2347] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/projects => generated 363 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:44.868841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:44.879507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe036-56c3-40a5-94a0-1ff4bf131358 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:44.943032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:44.943647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:44.943835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:44.944112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:44.944808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:44.946768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:44.955456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:45.036267 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1191/2348] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:44 2022] POST /identity/v3/users => generated 282 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:45.037773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.049022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db2f74-3b80-424a-8705-6cf21a0d8728 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.054280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.054981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.055263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.055562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.056217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.057866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.069359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1158/2349] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:45.073653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.085641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788362e8-7624-46ce-93f3-73f03b396c6f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.090892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba40a6f-bcb9-4328-9079-e159d1e329d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.091388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba40a6f-bcb9-4328-9079-e159d1e329d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.092463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba40a6f-bcb9-4328-9079-e159d1e329d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.092760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba40a6f-bcb9-4328-9079-e159d1e329d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.093529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba40a6f-bcb9-4328-9079-e159d1e329d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.095336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba40a6f-bcb9-4328-9079-e159d1e329d3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.118677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1192/2350] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:45.123350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.135635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e23eae5-d4ad-48cd-bdd3-e25a6bc07c63 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.142046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.142731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.143015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.143303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] PATH_INFO: `/v3/projects/e6c40353b4444e90a15a2df0ecabd0fe/users/c78b5c8d608444fc906c563fbc024774/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.163332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c40353b4444e90a15a2df0ecabd0fe, user_id=c78b5c8d608444fc906c563fbc024774, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.166308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.238928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1159/2351] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:45 2022] PUT /identity/v3/projects/e6c40353b4444e90a15a2df0ecabd0fe/users/c78b5c8d608444fc906c563fbc024774/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:16:45.251340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.251622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.251895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.282367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] MFA Rules not processed for user `c78b5c8d608444fc906c563fbc024774`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:45.283593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:45.352367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1193/2352] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:16:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:45.357060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.429079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9353da7-b843-4748-a3ca-76ea62897e77 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.433794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.434408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.434687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.434958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.436130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.438055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.538228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1160/2353] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:16:45 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:16:45.542743 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.555742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4550e8d-f8d2-492d-8ae8-d83d73f10abe None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.615994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.616623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.616908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.617187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.617855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.619827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.709238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1194/2354] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:16:45 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:16:45.713779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.730289 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b8297-4d2a-4ed7-9e5a-e80d6be73ee7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.735877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.736642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.737024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.737402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.738217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.739975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.755393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1161/2355] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:45.756594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.770074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d56cf5-fd2a-4345-a65d-a769b348356d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.775233 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.775897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.778015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.778015 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] PATH_INFO: `/v3/projects/a25ff029d6334e58a46bde964defff7f/users/89a0f01c87ca4902830482435606ec5e/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.792927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25ff029d6334e58a46bde964defff7f, user_id=89a0f01c87ca4902830482435606ec5e, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.795814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.873823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1195/2356] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:45 2022] PUT /identity/v3/projects/a25ff029d6334e58a46bde964defff7f/users/89a0f01c87ca4902830482435606ec5e/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:45.876162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:45.894537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1563d-3d7b-4219-9156-5f03cfaceba2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:45.971664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56827145-8779-4b51-ba2e-166fb0da6407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:45.972300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827145-8779-4b51-ba2e-166fb0da6407 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:45.972600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827145-8779-4b51-ba2e-166fb0da6407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:45.972876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827145-8779-4b51-ba2e-166fb0da6407 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:45.973527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827145-8779-4b51-ba2e-166fb0da6407 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:45.975364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827145-8779-4b51-ba2e-166fb0da6407 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:45.994936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1162/2357] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:16:46.002181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:46.014491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0e98c-c81f-4ec5-93fe-3a9ff538a5f9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:46.019642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:46.020333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:46.020619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:46.020894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] PATH_INFO: `/v3/projects/a25ff029d6334e58a46bde964defff7f/users/89a0f01c87ca4902830482435606ec5e/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:46.023530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a25ff029d6334e58a46bde964defff7f, user_id=89a0f01c87ca4902830482435606ec5e, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:46.026384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:46.047850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1196/2358] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:16:46 2022] PUT /identity/v3/projects/a25ff029d6334e58a46bde964defff7f/users/89a0f01c87ca4902830482435606ec5e/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:16:46.058362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:46.059870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:46.059870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:46.107009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] MFA Rules not processed for user `89a0f01c87ca4902830482435606ec5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:46.111256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:46.132870 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:46.192760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1163/2359] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:16:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:46.206016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:46.210561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:46.246817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:16:46.285593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85162cb5-a59e-46ac-9de1-f1d66aee5854 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:46.289526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec9e5-ed71-4026-a12c-3476b310f934 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:46.290332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec9e5-ed71-4026-a12c-3476b310f934 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:46.290898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec9e5-ed71-4026-a12c-3476b310f934 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:46.291172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec9e5-ed71-4026-a12c-3476b310f934 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:46.303657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec9e5-ed71-4026-a12c-3476b310f934 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:46.305048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec9e5-ed71-4026-a12c-3476b310f934 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:46.317719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1197/2360] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:46 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:46.549405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:46.564306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7096555-e265-4219-8808-fc46a94b4619 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:46.570617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:46.571263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:46.571551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:46.571834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:46.661617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:46.663347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:46.689012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1164/2361] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:51.602586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:51.603108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:51.603206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:51.624028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] MFA Rules not processed for user `942d175e7e2742728e66fea84c851038`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:16:51.625419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:51.693932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1198/2362] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:16:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:16:52.015069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.016557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:52.081264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e3ab5-94e8-4746-a3e9-79f7ab23150a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.094096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:52.094096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.094096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.094096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.094498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:52.098410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:52.099844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:52.108081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1165/2363] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:16:52.615684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:16:52.617192 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:52.696512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7edc8bf-571a-447c-be61-ffbf394c33b6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:16:52.703247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:16:52.703888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:16:52.704175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:16:52.704472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:16:52.706826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:16:52.719964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:16:52.721904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53be3b6-3836-4980-aa99-a60227ffd500 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:16:52.739141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1199/2364] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:16:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:00.069244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:00.137413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:00.143321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1837fa-a826-48e2-ab54-6392ac7a4c31 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:00.148551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:00.149407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.150037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.150373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] PATH_INFO: `/v3/users/c78b5c8d608444fc906c563fbc024774` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.151549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c78b5c8d608444fc906c563fbc024774)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:00.154141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:00.426216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c78b5c8d608444fc906c563fbc024774'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.433300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c78b5c8d608444fc906c563fbc024774'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.501964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c78b5c8d608444fc906c563fbc024774'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.502604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c78b5c8d608444fc906c563fbc024774'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.516073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c78b5c8d608444fc906c563fbc024774'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:00.522131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1166/2365] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:00 2022] DELETE /identity/v3/users/c78b5c8d608444fc906c563fbc024774 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:00.526771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.527238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.527348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.548648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:00.549902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:00.603423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:00.613487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1200/2366] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:00.623842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:00.625287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:00.711566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43c0f2-6f7a-4064-ae5c-048a24870e86 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:00.715263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:00.715906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:00.716196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:00.716470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:00.728684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:00.732209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:00.743701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1167/2367] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:01.173202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.174772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:01.187542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3496dee-b00a-4a7f-88a1-d1195dbaba3e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.192912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.193665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.194078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.194465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] PATH_INFO: `/v3/projects/e6c40353b4444e90a15a2df0ecabd0fe` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.195925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c40353b4444e90a15a2df0ecabd0fe)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.197959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.296125 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c40353b4444e90a15a2df0ecabd0fe', 'request_id': 'req-821f5069-c6aa-49a4-a3ff-99897550b2e1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.376177 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c40353b4444e90a15a2df0ecabd0fe', 'request_id': 'req-821f5069-c6aa-49a4-a3ff-99897550b2e1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.384531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c40353b4444e90a15a2df0ecabd0fe', 'request_id': 'req-821f5069-c6aa-49a4-a3ff-99897550b2e1'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.409543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] The token cache is being invalidate because project e6c40353b4444e90a15a2df0ecabd0fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:01.409901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.410593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.412295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1201/2368] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:01 2022] DELETE /identity/v3/projects/e6c40353b4444e90a15a2df0ecabd0fe => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:01.416991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:01.431512 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e273f-dfd7-41de-8422-358d7bfa8104 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:01.492241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:01.492888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:01.493175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:01.493480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] PATH_INFO: `/v3/users/89a0f01c87ca4902830482435606ec5e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:01.494718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a0f01c87ca4902830482435606ec5e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:01.496655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:01.521048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:01.732390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a0f01c87ca4902830482435606ec5e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.744121 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a0f01c87ca4902830482435606ec5e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.808323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a0f01c87ca4902830482435606ec5e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.809931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a0f01c87ca4902830482435606ec5e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.825855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d237a8fc-32fe-47d9-ae74-28c2640f1a67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a0f01c87ca4902830482435606ec5e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:01.832839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1168/2369] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:01 2022] DELETE /identity/v3/users/89a0f01c87ca4902830482435606ec5e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:02.241087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.243395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.261190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821f5069-c6aa-49a4-a3ff-99897550b2e1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.327468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.328195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.328406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.328702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] PATH_INFO: `/v3/projects/a25ff029d6334e58a46bde964defff7f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.330622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a25ff029d6334e58a46bde964defff7f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.332757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.448690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a25ff029d6334e58a46bde964defff7f', 'request_id': 'req-49d32202-a85b-4aca-824e-fe7a2aa1319d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.542314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a25ff029d6334e58a46bde964defff7f', 'request_id': 'req-49d32202-a85b-4aca-824e-fe7a2aa1319d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.550999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a25ff029d6334e58a46bde964defff7f', 'request_id': 'req-49d32202-a85b-4aca-824e-fe7a2aa1319d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.569777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] The token cache is being invalidate because project a25ff029d6334e58a46bde964defff7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:02.569777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.570523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-49d32202-a85b-4aca-824e-fe7a2aa1319d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:02.572396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1202/2370] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:02 2022] DELETE /identity/v3/projects/a25ff029d6334e58a46bde964defff7f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:02.577905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.578792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.579283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.606225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:02.608182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.687722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1169/2371] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:02.695528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.695900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.696250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.719141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:02.720374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:02.729046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1203/2372] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:02.733605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.743683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54c562-f3f6-4c47-af58-079af3358623 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.749569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.750272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.750563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.750848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.751513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:02.751821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.753445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.766906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1170/2373] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:17:02.771530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.778837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:02.782945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460ed1b1-8340-419c-b6ef-147145eee156 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.797109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.797109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.797109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.797109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.797498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.797498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:02.910037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1204/2374] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:02 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:02.914446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:02.925501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6433fe-4f5b-42f2-b58f-9a6499ccc4c7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:02.982267 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:02.983423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:02.983703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:02.983976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:02.984604 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:02.986542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.068896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1171/2375] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:02 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:03.073373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.084865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b936f1e-0af9-476c-bc77-bc03e9c200e0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.090086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.090697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.090982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.091386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.092105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.093811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.104860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1205/2376] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:03.110247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.126567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438768f6-8e31-4758-9673-aaff4dcfb893 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.134172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62a1ec1-0259-4e7e-aa2b-8091b0356f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.135199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a1ec1-0259-4e7e-aa2b-8091b0356f81 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.135693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a1ec1-0259-4e7e-aa2b-8091b0356f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.136495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a1ec1-0259-4e7e-aa2b-8091b0356f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.137338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a1ec1-0259-4e7e-aa2b-8091b0356f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.139188 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62a1ec1-0259-4e7e-aa2b-8091b0356f81 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.161621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1172/2377] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:03.165611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.178659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f25514-ebbe-4a34-82f7-21c43599b83b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.183541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.184214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.184503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.184911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] PATH_INFO: `/v3/projects/b387196a110b4bf8aeb570341e94f766/users/cce68eb525694fed9ecfab8cc1a86d71/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.202606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b387196a110b4bf8aeb570341e94f766, user_id=cce68eb525694fed9ecfab8cc1a86d71, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.205682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.314437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1206/2378] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:03 2022] PUT /identity/v3/projects/b387196a110b4bf8aeb570341e94f766/users/cce68eb525694fed9ecfab8cc1a86d71/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:03.326235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.326471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.326750 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.334765 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:03.355041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] MFA Rules not processed for user `cce68eb525694fed9ecfab8cc1a86d71`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:03.356253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:03.421977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1173/2379] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:03.426604 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.490963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8462929-b3f5-496b-85aa-2391ebccd066 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.496250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.496875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.497157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.497431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.498150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.500176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.583967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1207/2380] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:03 2022] POST /identity/v3/projects => generated 387 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:03.588104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.600099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332368ec-7271-473e-8960-bb6ca3b5dfc3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.666749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.667368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.667654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.667930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.668575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.670587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.796623 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1174/2381] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:03 2022] POST /identity/v3/users => generated 300 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:03.801353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.813943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2760a6f0-40ec-42c5-9670-16b545aea77b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.819460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.820379 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.821671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.821671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.821671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.823623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.836439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1208/2382] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:03.840911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.852914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4104215-5902-49f0-87f9-9a9149e572cf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:03.858442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:03.859362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:03.859710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:03.860304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] PATH_INFO: `/v3/projects/bf8d89c736c34c85bc18d1d1ea25db7c/users/5707574da5a24a7599eaa2a1a04b2aa3/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:03.893553 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8d89c736c34c85bc18d1d1ea25db7c, user_id=5707574da5a24a7599eaa2a1a04b2aa3, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:03.896382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:03.979811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1175/2383] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:03 2022] PUT /identity/v3/projects/bf8d89c736c34c85bc18d1d1ea25db7c/users/5707574da5a24a7599eaa2a1a04b2aa3/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:03.984364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:03.994620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc255e-f1f2-4df2-b55f-3ecd8f389fae admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:04.058026 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f16afb-a669-4230-a20d-fc691808aa80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:04.058647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16afb-a669-4230-a20d-fc691808aa80 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.058968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16afb-a669-4230-a20d-fc691808aa80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.059249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f16afb-a669-4230-a20d-fc691808aa80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.059896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f16afb-a669-4230-a20d-fc691808aa80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:04.061687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f16afb-a669-4230-a20d-fc691808aa80 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.072703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1209/2384] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:04.076747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:04.091322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5c5f-3889-4e32-92e9-9ed85f1bb7e4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:04.094210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:04.094210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.094210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.094210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] PATH_INFO: `/v3/projects/bf8d89c736c34c85bc18d1d1ea25db7c/users/5707574da5a24a7599eaa2a1a04b2aa3/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.096128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf8d89c736c34c85bc18d1d1ea25db7c, user_id=5707574da5a24a7599eaa2a1a04b2aa3, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:04.098814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.110359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1176/2385] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:04 2022] PUT /identity/v3/projects/bf8d89c736c34c85bc18d1d1ea25db7c/users/5707574da5a24a7599eaa2a1a04b2aa3/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:04.120872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.121161 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.121522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.150769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] MFA Rules not processed for user `5707574da5a24a7599eaa2a1a04b2aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:04.151987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.157673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:04.221898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1210/2386] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:04.233407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:04.235298 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.287977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:04.293614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef824d8-bc23-413c-aa5d-710417ec93ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:04.298400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:04.299043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.299328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.299606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.308952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:04.310416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.320338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1177/2387] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:04.705868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:04.712302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.788041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b2557-970e-449c-8f9b-eff3e1104672 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:04.793958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:04.794573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:04.794881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:04.795160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:04.796980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:04.813030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:04.814774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0633840a-415e-48b2-8590-9fad5ce536de service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:04.825999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1211/2388] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:05.729519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:05.809242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f368f60-a746-4e30-87b6-c5c95cd76aa6 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:05.816084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:05.816956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:05.817245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:05.817528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] PATH_INFO: `/v3/users/cce68eb525694fed9ecfab8cc1a86d71` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:05.818735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cce68eb525694fed9ecfab8cc1a86d71)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:05.820698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.155489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cce68eb525694fed9ecfab8cc1a86d71'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.167505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cce68eb525694fed9ecfab8cc1a86d71'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.256474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cce68eb525694fed9ecfab8cc1a86d71'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.257208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cce68eb525694fed9ecfab8cc1a86d71'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.275473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cce68eb525694fed9ecfab8cc1a86d71'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:06.293607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1178/2389] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:05 2022] DELETE /identity/v3/users/cce68eb525694fed9ecfab8cc1a86d71 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:06.298900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.299441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.299441 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.319413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:06.320628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.383435 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1212/2390] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:06.393978 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.395464 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.460548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52663c-88a4-4a4a-98be-dcf8959bfb8c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.467199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.468764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.468764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.468764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.478786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.480251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.491689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1179/2391] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:06.886926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:06.888348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:06.900541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02defa66-4b8a-4638-a8f0-ad1250483414 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:06.905288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:06.905968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:06.906248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:06.906526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] PATH_INFO: `/v3/projects/b387196a110b4bf8aeb570341e94f766` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:06.907864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b387196a110b4bf8aeb570341e94f766)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:06.909823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:06.939050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:07.051007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b387196a110b4bf8aeb570341e94f766', 'request_id': 'req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.158098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b387196a110b4bf8aeb570341e94f766', 'request_id': 'req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.166122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b387196a110b4bf8aeb570341e94f766', 'request_id': 'req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.177991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] The token cache is being invalidate because project b387196a110b4bf8aeb570341e94f766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:07.178285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.178938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.180408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1213/2392] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:06 2022] DELETE /identity/v3/projects/b387196a110b4bf8aeb570341e94f766 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:07.184487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:07.195709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4fada1-ff5a-4689-9f17-69bdd2149979 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:07.204664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:07.255996 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:07.256635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:07.256932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:07.257199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] PATH_INFO: `/v3/users/5707574da5a24a7599eaa2a1a04b2aa3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:07.258343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5707574da5a24a7599eaa2a1a04b2aa3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:07.260268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:07.500681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5707574da5a24a7599eaa2a1a04b2aa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.508370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5707574da5a24a7599eaa2a1a04b2aa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.621510 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5707574da5a24a7599eaa2a1a04b2aa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.622170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5707574da5a24a7599eaa2a1a04b2aa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.636001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c4464bb-ef93-4fdb-86f3-4f5e87c7a72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5707574da5a24a7599eaa2a1a04b2aa3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:07.643332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1180/2393] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:07 2022] DELETE /identity/v3/users/5707574da5a24a7599eaa2a1a04b2aa3 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:08.132495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:08.134098 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:08.157879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f08f5-20d4-46f3-9dbb-1bbd6828d828 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:08.252713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:08.253352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.253644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.253949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] PATH_INFO: `/v3/projects/bf8d89c736c34c85bc18d1d1ea25db7c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.255078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf8d89c736c34c85bc18d1d1ea25db7c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:08.257034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:08.379948 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf8d89c736c34c85bc18d1d1ea25db7c', 'request_id': 'req-a429284b-e9ef-40e5-987f-be1d3a424919'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.486516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf8d89c736c34c85bc18d1d1ea25db7c', 'request_id': 'req-a429284b-e9ef-40e5-987f-be1d3a424919'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.495206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf8d89c736c34c85bc18d1d1ea25db7c', 'request_id': 'req-a429284b-e9ef-40e5-987f-be1d3a424919'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.510394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] The token cache is being invalidate because project bf8d89c736c34c85bc18d1d1ea25db7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:08.510773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.511580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a429284b-e9ef-40e5-987f-be1d3a424919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:08.513268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1214/2394] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:08 2022] DELETE /identity/v3/projects/bf8d89c736c34c85bc18d1d1ea25db7c => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:08.519816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.520387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.520841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.544920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:08.546367 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:08.611375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1181/2395] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:08.620864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.621351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.621575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.644184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:08.645431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:08.653912 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1215/2396] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:08.658980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:08.669700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b42d4f-7638-4999-b8a8-d8e4db2ecf17 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:08.675331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:08.675962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.676274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.676557 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.677220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:08.677529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:08.679239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:08.693335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1182/2397] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:17:08.698014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:08.708844 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fae503-a22c-4f7f-b6ec-80f74f9246fe None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:08.713622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:08.714255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.714532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.714811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.715582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:08.717528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:08.840040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1216/2398] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:08 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:08.841719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:08.853710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c3e84b-968d-4f83-b8ac-2dd5c2e08b32 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:08.868581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:08.923787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:08.923787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:08.923787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:08.923787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:08.924637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:08.925279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.036276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1183/2399] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:08 2022] POST /identity/v3/users => generated 290 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:09.037657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.049030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e586a80-d699-422e-9260-fcdfd45370d6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.053823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.054957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.055246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.055540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.056249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.057941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.072929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1217/2400] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:09.074387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.086451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe2d029-9aab-4231-acbd-feb55c73b911 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.091729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.092347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.092637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.092920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.093599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.095308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.111718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1184/2401] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:09.112182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.117867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:09.123690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c882100-b4c3-4afe-b99e-7accc9054ddd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.130228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c66eaed-ee2e-4849-abdb-14943f6136dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.131519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66eaed-ee2e-4849-abdb-14943f6136dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.132745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66eaed-ee2e-4849-abdb-14943f6136dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.132745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c66eaed-ee2e-4849-abdb-14943f6136dc admin admin] PATH_INFO: `/v3/projects/ea8294141f3047418ccea9f29e699cda/users/24e98858511a43f587fe8b9d7d1d2cf4/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.157375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66eaed-ee2e-4849-abdb-14943f6136dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8294141f3047418ccea9f29e699cda, user_id=24e98858511a43f587fe8b9d7d1d2cf4, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.160201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c66eaed-ee2e-4849-abdb-14943f6136dc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.189572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.191466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.253999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1218/2402] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:09 2022] PUT /identity/v3/projects/ea8294141f3047418ccea9f29e699cda/users/24e98858511a43f587fe8b9d7d1d2cf4/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:09.262075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.262495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.263017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.281582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198457e-83b2-41fe-820e-efe55ab23468 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.296955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] MFA Rules not processed for user `24e98858511a43f587fe8b9d7d1d2cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:09.298652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.357373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.358069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.358321 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.358599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.360947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:09.375606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.377288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.377884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1219/2403] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:09.379861 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.389372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1185/2404] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:09.456046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991bc2c9-228e-4c1b-a394-e009c95aefa2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.460877 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.461501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.461807 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.462087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.462724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.465359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.556536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1220/2405] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:09.560377 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.571895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea620d-18b7-414f-a5e5-e7975dfcd9d5 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.590924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:09.638814 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.639436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.639718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.640026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.641397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.643461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.769425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1186/2406] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:09 2022] POST /identity/v3/users => generated 296 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:09.770549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.781977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32981bc-2edc-43e8-9352-322a4f57e5da admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.786765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.791777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.791777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.791777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.791777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.791777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.803999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1221/2407] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:09.808075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.818986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afea66e-2441-48ec-85f5-854bf88193c6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:09.824848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:09.824848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:09.824848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:09.825219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] PATH_INFO: `/v3/projects/a2a60549d39d4111ac0133a74f687e61/users/63d43c38c45e4372b04013bfd5739c13/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:09.850156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a60549d39d4111ac0133a74f687e61, user_id=63d43c38c45e4372b04013bfd5739c13, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:09.853438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:09.956053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1187/2408] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:09 2022] PUT /identity/v3/projects/a2a60549d39d4111ac0133a74f687e61/users/63d43c38c45e4372b04013bfd5739c13/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:09.960258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:09.976745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973e6ec-3bc6-4861-a98a-d8d9e356edee admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.030224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:10.052121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9b608-a878-440a-9545-bd85063c2281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.052902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9b608-a878-440a-9545-bd85063c2281 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.053964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9b608-a878-440a-9545-bd85063c2281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.053964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9b608-a878-440a-9545-bd85063c2281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.054789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9b608-a878-440a-9545-bd85063c2281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.056672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9b608-a878-440a-9545-bd85063c2281 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.080351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1222/2409] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:10.081593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.104012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b9479f-c19b-4301-a2b2-c013bc1cf822 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.109547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.110847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.112092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.113562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] PATH_INFO: `/v3/projects/a2a60549d39d4111ac0133a74f687e61/users/63d43c38c45e4372b04013bfd5739c13/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.116462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a60549d39d4111ac0133a74f687e61, user_id=63d43c38c45e4372b04013bfd5739c13, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.119478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.134866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1188/2410] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:10 2022] PUT /identity/v3/projects/a2a60549d39d4111ac0133a74f687e61/users/63d43c38c45e4372b04013bfd5739c13/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:10.146852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.147423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.147769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.205008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] MFA Rules not processed for user `63d43c38c45e4372b04013bfd5739c13`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:10.206882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:10.294292 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1223/2411] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:10 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:10.307092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.308863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:10.379462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73667cf0-32c2-428d-8f9e-078c05579e7b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.383564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.384202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.384483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.384760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.395644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.399083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.408729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1189/2412] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:10.552551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:10.563932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb52ed4a-13e0-4521-9f9f-011abd16932d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:10.567856 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1205f-0e5c-4a01-bb9e-f53c01befd29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:10.568471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1205f-0e5c-4a01-bb9e-f53c01befd29 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:10.568751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1205f-0e5c-4a01-bb9e-f53c01befd29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:10.569025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1205f-0e5c-4a01-bb9e-f53c01befd29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:10.629066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1205f-0e5c-4a01-bb9e-f53c01befd29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:10.630522 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1205f-0e5c-4a01-bb9e-f53c01befd29 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:10.639621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1224/2413] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:11.145821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.210407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb55b32-71e1-4305-aeb4-11b684216c32 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.215326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.215975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.216253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.216527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] PATH_INFO: `/v3/users/24e98858511a43f587fe8b9d7d1d2cf4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.217662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24e98858511a43f587fe8b9d7d1d2cf4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.219687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.238837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:11.494127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24e98858511a43f587fe8b9d7d1d2cf4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.501237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24e98858511a43f587fe8b9d7d1d2cf4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.563247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24e98858511a43f587fe8b9d7d1d2cf4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.563895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24e98858511a43f587fe8b9d7d1d2cf4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.579566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24e98858511a43f587fe8b9d7d1d2cf4'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:11.585883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1190/2414] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:11 2022] DELETE /identity/v3/users/24e98858511a43f587fe8b9d7d1d2cf4 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:11.590689 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.591186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.591303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.611806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:11.613017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:11.672898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1225/2415] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:11.688709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:11.688709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:11.751083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7502b38-2365-4ad8-911b-0adf9ea60d51 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:11.754748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:11.755363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:11.755653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:11.755932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:11.767179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:11.768631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:11.778548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1191/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:12.212322 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.214366 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.224642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.232560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfc8a8-19c6-4247-8c58-efba40babf98 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.242228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.243793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.243793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.243793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] PATH_INFO: `/v3/projects/ea8294141f3047418ccea9f29e699cda` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.244955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8294141f3047418ccea9f29e699cda)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.247357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.402927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8294141f3047418ccea9f29e699cda', 'request_id': 'req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.518710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8294141f3047418ccea9f29e699cda', 'request_id': 'req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.526545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8294141f3047418ccea9f29e699cda', 'request_id': 'req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.541201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] The token cache is being invalidate because project ea8294141f3047418ccea9f29e699cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:12.541541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.542498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-57482cac-c2f0-4f97-a6da-9fb1f4cef6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.544154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1226/2417] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:12 2022] DELETE /identity/v3/projects/ea8294141f3047418ccea9f29e699cda => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:12.548196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.564781 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33423cbc-b91f-4f12-b1be-388ff6828ad1 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.575075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.575361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.575635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.596631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:12.597878 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.642332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.643515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.643803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.644141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] PATH_INFO: `/v3/users/63d43c38c45e4372b04013bfd5739c13` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.645315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d43c38c45e4372b04013bfd5739c13)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.647355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.666814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1227/2418] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:17:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:12.713848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:12.715194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:12.788016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd03591-cbd8-4515-9af9-6f9339a14247 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:12.793475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:12.794118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:12.794407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:12.794681 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:12.803641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:12.805388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:12.815669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1228/2419] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:17:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:17:12.853262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:12.866633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d43c38c45e4372b04013bfd5739c13'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.874398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d43c38c45e4372b04013bfd5739c13'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.941994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d43c38c45e4372b04013bfd5739c13'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.942768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d43c38c45e4372b04013bfd5739c13'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.958643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ca990ebb-a3ed-4c6e-98e3-9245324b8fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d43c38c45e4372b04013bfd5739c13'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:12.965483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1192/2420] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:12 2022] DELETE /identity/v3/users/63d43c38c45e4372b04013bfd5739c13 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:13.836078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:13.841105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:13.867338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc9a64e-9a4a-4af1-b3d9-100f83808c75 service swift] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:13.888919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:13.952457 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:13.953279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:13.953553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:13.953892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] PATH_INFO: `/v3/projects/a2a60549d39d4111ac0133a74f687e61` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:13.955384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a60549d39d4111ac0133a74f687e61)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:13.957466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.090772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a60549d39d4111ac0133a74f687e61', 'request_id': 'req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.170597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a60549d39d4111ac0133a74f687e61', 'request_id': 'req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.180004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a60549d39d4111ac0133a74f687e61', 'request_id': 'req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.204141 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] The token cache is being invalidate because project a2a60549d39d4111ac0133a74f687e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:14.204640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.205407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b1d1b39f-57a6-45e8-b6f4-97a8f9e81708 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:14.210640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1229/2421] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:13 2022] DELETE /identity/v3/projects/a2a60549d39d4111ac0133a74f687e61 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:14.212263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.212756 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.212940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.238457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:14.239702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.336484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1193/2422] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:14.405138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.405463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.407494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.431247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:14.432557 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:14.448516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.453956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1230/2423] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:14.464592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b070cc3-46ae-4947-934b-fc843f92f77a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.479217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.479946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.480232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.480508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.481169 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:14.481480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.488845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.505109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1194/2424] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:14.511563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.525864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d74fd26-ee42-4956-8a46-e84523ba450b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.532405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.533200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.533699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.534370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.535377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.542220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.674945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1231/2425] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:14.679205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.697899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a738e5d-6f34-484b-ba29-ae056aa8b93f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.813065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.813876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.814227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.814607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.815375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.818005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:14.953371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1195/2426] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:14 2022] POST /identity/v3/users => generated 288 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:14.957721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:14.975115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920c6b1-da5e-4229-a661-032f407465fa admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:14.988723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:14.988723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:14.988723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:14.988723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:14.989327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:14.995785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.015953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1232/2427] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:14 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:15.020373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.037795 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2eea6-2671-4e5b-9aca-2917fd37f5c6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.046146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf214a5-6167-44e4-b758-2b7187c2c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.048049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf214a5-6167-44e4-b758-2b7187c2c276 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.048202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf214a5-6167-44e4-b758-2b7187c2c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.048573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf214a5-6167-44e4-b758-2b7187c2c276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.049314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf214a5-6167-44e4-b758-2b7187c2c276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.052408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf214a5-6167-44e4-b758-2b7187c2c276 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.074658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1196/2428] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:15.079279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.102497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7d640-a4ed-4fd3-a908-ae7dfbe591b9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.110626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.111427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.111816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.112199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] PATH_INFO: `/v3/projects/ff4feca37bcd409090eed4f1c9a91f78/users/11b64251bff640ec996c14d81585f3ae/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.131981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff4feca37bcd409090eed4f1c9a91f78, user_id=11b64251bff640ec996c14d81585f3ae, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.134837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.204913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1233/2429] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:15 2022] PUT /identity/v3/projects/ff4feca37bcd409090eed4f1c9a91f78/users/11b64251bff640ec996c14d81585f3ae/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:15.215522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1fc4b-76cf-4617-a8cc-5d3ead63cd23 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.216084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1fc4b-76cf-4617-a8cc-5d3ead63cd23 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.216084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf1fc4b-76cf-4617-a8cc-5d3ead63cd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.249658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cdf1fc4b-76cf-4617-a8cc-5d3ead63cd23 None None] MFA Rules not processed for user `11b64251bff640ec996c14d81585f3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:15.251040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cdf1fc4b-76cf-4617-a8cc-5d3ead63cd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.264305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:15.332071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1197/2430] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:15.345564 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:15.347411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:15.411010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:15.420223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9faa27a-56b0-4855-8c2e-51c77677061a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:15.425018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a4a9a-6a00-444f-baec-de7c24f23e1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:15.425649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a4a9a-6a00-444f-baec-de7c24f23e1c service glance] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:15.425967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a4a9a-6a00-444f-baec-de7c24f23e1c service glance] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:15.431334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a4a9a-6a00-444f-baec-de7c24f23e1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:15.437946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a4a9a-6a00-444f-baec-de7c24f23e1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:15.440076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a4a9a-6a00-444f-baec-de7c24f23e1c service glance] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:15.450920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1234/2431] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:25.401905 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=22 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:25.548459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:32.735766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b2d8d-b0a1-489c-82ad-652273f26b5a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:32.736309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b2d8d-b0a1-489c-82ad-652273f26b5a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:32.736309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604b2d8d-b0a1-489c-82ad-652273f26b5a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:32.737649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1198/2432] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:17:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:17:32.744254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d842262-5bfb-4255-a853-52301e4641a2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:32.744695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d842262-5bfb-4255-a853-52301e4641a2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:32.744837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d842262-5bfb-4255-a853-52301e4641a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:32.765444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2d842262-5bfb-4255-a853-52301e4641a2 None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:32.766699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2d842262-5bfb-4255-a853-52301e4641a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:32.838377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1235/2433] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:17:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:32.850932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b2d8d-b0a1-489c-82ad-652273f26b5a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:32.852699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-604b2d8d-b0a1-489c-82ad-652273f26b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:32.869761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604b2d8d-b0a1-489c-82ad-652273f26b5a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:32.873957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f82c99e-791f-45ff-bbc2-9801151b4c97 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:32.875086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c99e-791f-45ff-bbc2-9801151b4c97 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:32.875366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c99e-791f-45ff-bbc2-9801151b4c97 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:32.875647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f82c99e-791f-45ff-bbc2-9801151b4c97 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:32.887303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82c99e-791f-45ff-bbc2-9801151b4c97 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:32.889069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f82c99e-791f-45ff-bbc2-9801151b4c97 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:32.906218 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1199/2434] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:33.159302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f7a78-0fa3-4175-b093-7fd66f159949 None None] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:33.159645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f7a78-0fa3-4175-b093-7fd66f159949 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:33.159979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8f7a78-0fa3-4175-b093-7fd66f159949 None None] PATH_INFO: `/` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:33.161374 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1236/2435] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:17:33.166687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863e944-6811-4ed8-ab92-9a87263fd863 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:33.166979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863e944-6811-4ed8-ab92-9a87263fd863 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:33.167262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8863e944-6811-4ed8-ab92-9a87263fd863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:33.187925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8863e944-6811-4ed8-ab92-9a87263fd863 None None] MFA Rules not processed for user `851cf4bd4efd448f8fc04563775de1ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:33.189123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8863e944-6811-4ed8-ab92-9a87263fd863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:33.240858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1200/2436] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:33.253581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f7a78-0fa3-4175-b093-7fd66f159949 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:33.256122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a8f7a78-0fa3-4175-b093-7fd66f159949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:33.269066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8f7a78-0fa3-4175-b093-7fd66f159949 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:33.278332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10abc3e5-389e-4baa-b498-e205521b448d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:33.278955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc3e5-389e-4baa-b498-e205521b448d service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:33.279237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc3e5-389e-4baa-b498-e205521b448d service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:33.279512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10abc3e5-389e-4baa-b498-e205521b448d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:33.288051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abc3e5-389e-4baa-b498-e205521b448d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:33.289779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10abc3e5-389e-4baa-b498-e205521b448d service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:33.297595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1237/2437] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:33.382050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d94bfe-daaf-4738-8c68-ba7a1bf124f8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:33.382340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d94bfe-daaf-4738-8c68-ba7a1bf124f8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:33.382621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d94bfe-daaf-4738-8c68-ba7a1bf124f8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:33.384111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1201/2438] 10.222.0.52 () {54 vars in 921 bytes} [Tue Aug 30 01:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 01:17:33.389826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:33.391027 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:33.391451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:33.413187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] MFA Rules not processed for user `e1a623f1fc434449a76106e74756e35c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:33.414622 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:33.424650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1238/2439] 10.222.0.52 () {58 vars in 1018 bytes} [Tue Aug 30 01:17:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:33.435199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d94bfe-daaf-4738-8c68-ba7a1bf124f8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:33.447965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d94bfe-daaf-4738-8c68-ba7a1bf124f8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:33.451998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:33.453139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:33.453458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:33.453893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:33.465323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:33.466882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:33.479647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1202/2440] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:35.299922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:35.370350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a1a2f-9871-41c6-85f8-cc7dac5b3f8c None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:35.376609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:35.377357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.377749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.378101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] PATH_INFO: `/v3/users/11b64251bff640ec996c14d81585f3ae` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.379442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b64251bff640ec996c14d81585f3ae)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:35.381827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:35.447237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:35.457575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddde9b-23d7-4f4b-9ac0-eb9b6d2b9785 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:35.461381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:35.462033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.462343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.462683 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.468656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:35.478775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:35.480542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37def5f0-b723-434b-a963-162f76dcbbfd service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:35.492112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1203/2441] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:35.599816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b64251bff640ec996c14d81585f3ae'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.607351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b64251bff640ec996c14d81585f3ae'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.608160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b64251bff640ec996c14d81585f3ae'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.609440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b64251bff640ec996c14d81585f3ae'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.622623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b64251bff640ec996c14d81585f3ae'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:35.624945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:35.731462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1239/2442] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:35 2022] DELETE /identity/v3/users/11b64251bff640ec996c14d81585f3ae => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:35.736721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.737011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.737288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.757584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:35.758866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:35.822715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1204/2443] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:35.832527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:35.833998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:35.895472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97fc5a-8365-492f-810a-637208763144 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:35.899204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7574917-f594-49ee-8ba1-a8f559f80ae9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:35.899759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7574917-f594-49ee-8ba1-a8f559f80ae9 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:35.900043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7574917-f594-49ee-8ba1-a8f559f80ae9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:35.900428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7574917-f594-49ee-8ba1-a8f559f80ae9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:35.910439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7574917-f594-49ee-8ba1-a8f559f80ae9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:35.911884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7574917-f594-49ee-8ba1-a8f559f80ae9 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:35.922079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1240/2444] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:36.463192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:36.470250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:36.480174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92f8ee-f34b-4bfa-9db8-52232e67a844 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:36.501772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:36.503747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.503747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.503747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] PATH_INFO: `/v3/projects/ff4feca37bcd409090eed4f1c9a91f78` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.504519 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff4feca37bcd409090eed4f1c9a91f78)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:36.506502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:36.636605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff4feca37bcd409090eed4f1c9a91f78', 'request_id': 'req-adcf2696-b970-4ecd-9577-6d3bda4cdac7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.645106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff4feca37bcd409090eed4f1c9a91f78', 'request_id': 'req-adcf2696-b970-4ecd-9577-6d3bda4cdac7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.660476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff4feca37bcd409090eed4f1c9a91f78', 'request_id': 'req-adcf2696-b970-4ecd-9577-6d3bda4cdac7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.661386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:36.744270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] The token cache is being invalidate because project ff4feca37bcd409090eed4f1c9a91f78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:36.744582 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.745207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-adcf2696-b970-4ecd-9577-6d3bda4cdac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:36.746742 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1205/2445] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:36 2022] DELETE /identity/v3/projects/ff4feca37bcd409090eed4f1c9a91f78 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:36.752319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.753111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.753111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.775392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:36.776132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:36.852854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1241/2446] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:36.861603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.862000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.862283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.887473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:36.888723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:36.897661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1206/2447] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:36.902352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:36.914094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25453f4d-2c85-40d0-8203-6a4e68ed6b94 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:36.919343 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:36.919964 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.922319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.922319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.922319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:36.922319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:36.924308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd40095-79bb-4b6c-aaff-6cb32654a248 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:36.936235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1242/2448] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:36.940835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:36.950600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0b088-cc0a-4575-afe9-dff6bb2a7ee5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:36.955877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:36.956593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:36.957006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:36.957328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:36.958077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:36.960082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.050031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.050344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.050628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.072037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:37.073384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.079406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1207/2449] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:36 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:37.083291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.137813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1243/2450] 10.222.0.52 () {58 vars in 1009 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:37.151207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.152561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.157777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cb5521-e924-40dd-8a4c-be4494c51e3f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.160895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.161498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.161807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.162083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.162710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.164934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.185308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:37.226195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f4a78-de14-4142-92f6-a9faaa636f42 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.279583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1208/2451] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:37.280828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.293109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca660b54-9bf1-4ce2-8b63-f58fe43e7e0e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.300213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.300984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.301282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.302062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.302411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.304173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.315796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e34a3a-fb47-4f6d-99e9-bd9454a3c99d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.316731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1209/2452] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:37.317989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34a3a-fb47-4f6d-99e9-bd9454a3c99d service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.318303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34a3a-fb47-4f6d-99e9-bd9454a3c99d service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.318590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e34a3a-fb47-4f6d-99e9-bd9454a3c99d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.320426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.338892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2148b-af7d-4a34-b51c-9940aa297a37 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.346967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.347944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.349134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.349629 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.350999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.353848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.373986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1210/2453] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:37.376495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.387500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221a869-92a5-4c0d-b0b0-6e7e3d90dca4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.392673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.393516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.393927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.394405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] PATH_INFO: `/v3/projects/4cb7ea861caf4e0e9047dab83da411a4/users/ef39008f78bd4c6f9547b35de6738c77/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.403848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e34a3a-fb47-4f6d-99e9-bd9454a3c99d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.406591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e34a3a-fb47-4f6d-99e9-bd9454a3c99d service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.412890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb7ea861caf4e0e9047dab83da411a4, user_id=ef39008f78bd4c6f9547b35de6738c77, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.416236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.419514 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1244/2454] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:17:37.512908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1211/2455] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:37 2022] PUT /identity/v3/projects/4cb7ea861caf4e0e9047dab83da411a4/users/ef39008f78bd4c6f9547b35de6738c77/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:37.530735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.531162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.531550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.561409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] MFA Rules not processed for user `ef39008f78bd4c6f9547b35de6738c77`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:37.563623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.707402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:37.719160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1245/2456] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:37.736276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:37.738495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:37.937896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601d1c61-d5b6-45ce-a7e6-f6012b549ef4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:37.937896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:37.937896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:37.937896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:37.939361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:37.939361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:37.939361 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:37.974250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1212/2457] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:37 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:41.195651 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:41.198202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:41.285303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8966cf79-1cbb-4b70-bf4e-4c2fc8714afa None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:41.290761 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:41.291700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:41.292106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:41.292488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] PATH_INFO: `/v3/users/90486e175bb2411b9f2adedf6e3ecac6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:41.293885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] RBAC: Authorizing `identity:delete_user(user_id=90486e175bb2411b9f2adedf6e3ecac6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:41.296751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:42.261545 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90486e175bb2411b9f2adedf6e3ecac6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.269680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90486e175bb2411b9f2adedf6e3ecac6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.270452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90486e175bb2411b9f2adedf6e3ecac6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.271084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90486e175bb2411b9f2adedf6e3ecac6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.294814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e1083d19-3039-42dd-b891-0c0e9e96adac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90486e175bb2411b9f2adedf6e3ecac6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.382549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1246/2458] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:41 2022] DELETE /identity/v3/users/90486e175bb2411b9f2adedf6e3ecac6 => generated 0 bytes in 1188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:42.673612 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:42.683304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66c062f-5ad3-40fe-a8f5-075a235b1bfe service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:42.692840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:42.753503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:42.754276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:42.754556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:42.754766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] PATH_INFO: `/v3/projects/13862ab3a9504644853a8c48c3f69b4c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:42.755965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=13862ab3a9504644853a8c48c3f69b4c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:42.758108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:42.880190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13862ab3a9504644853a8c48c3f69b4c', 'request_id': 'req-f17e3207-4310-449f-84e3-d0e7e0d5da7c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.887883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13862ab3a9504644853a8c48c3f69b4c', 'request_id': 'req-f17e3207-4310-449f-84e3-d0e7e0d5da7c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.895303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13862ab3a9504644853a8c48c3f69b4c', 'request_id': 'req-f17e3207-4310-449f-84e3-d0e7e0d5da7c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.977644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] The token cache is being invalidate because project 13862ab3a9504644853a8c48c3f69b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:42.977982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.978655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f17e3207-4310-449f-84e3-d0e7e0d5da7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:42.980571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1213/2459] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:42 2022] DELETE /identity/v3/projects/13862ab3a9504644853a8c48c3f69b4c => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:42.986536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:42.986838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:42.987226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.009372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:43.023044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.099504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1247/2460] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:43.107256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.107543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.107826 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.128222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:43.129468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:43.137503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1214/2461] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:43.142654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.153360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff883cf-02c0-44eb-a8cb-ed40625f63d6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.161089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.161722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.162035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.162310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.163364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:43.163570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.165465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.183154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1248/2462] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:17:43.185027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.197277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c953a4-db61-4821-a2ed-ea8325c45d0c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.203565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.204190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.204475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.204757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.205660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.209715 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.316874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1215/2463] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:43 2022] POST /identity/v3/projects => generated 371 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:43.323342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.344924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0f87a-4719-47f1-a70f-0d817110b6c2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.361554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:43.416352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.417088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.417459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.417819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.418527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.420538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.528840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1249/2464] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:43 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:43.534203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.552982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4b985-0fd3-4c0a-915f-9d8bc3f56701 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.562297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.563490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.567476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.567476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.567476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.568695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.589788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1216/2465] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:43.593710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.604284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e19960a-e4be-46b3-aa8d-aacd8bd9dc17 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.610115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d103f-cb8d-4747-9d01-e9345794fe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.610902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d103f-cb8d-4747-9d01-e9345794fe14 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.611185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d103f-cb8d-4747-9d01-e9345794fe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.611471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d103f-cb8d-4747-9d01-e9345794fe14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.612301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d103f-cb8d-4747-9d01-e9345794fe14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.614325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d103f-cb8d-4747-9d01-e9345794fe14 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.627520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1250/2466] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:43.637110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:43.649053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba604bf-537e-4c62-b53d-5225d40e9c51 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:43.657505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:43.658402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.664442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.664442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] PATH_INFO: `/v3/projects/630a924648ac4094a731c69c96212e0c/users/e682fbdcbec14ea3bbdc02d73d1b981c/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.676578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=630a924648ac4094a731c69c96212e0c, user_id=e682fbdcbec14ea3bbdc02d73d1b981c, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:43.679294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:43.895556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1217/2467] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:43 2022] PUT /identity/v3/projects/630a924648ac4094a731c69c96212e0c/users/e682fbdcbec14ea3bbdc02d73d1b981c/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:43.906795 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:43.907119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:43.907357 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:43.941583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] MFA Rules not processed for user `e682fbdcbec14ea3bbdc02d73d1b981c`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:43.942868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:44.019354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1251/2468] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:44.035932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.092397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:44.123371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dfb608-633d-4210-8cd0-2ba32ea86160 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.128588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.129221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.129532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.129842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.130513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.132429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.314477 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1218/2469] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/projects => generated 373 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:44.318677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.329745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfececbe-e001-4b7c-89cd-e64bc919c9a3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.394263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.394897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.395605 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.395896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.397683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.399978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.604977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1252/2470] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:44 2022] POST /identity/v3/users => generated 293 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:44.609202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.620555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c2cce3-348e-45e2-ac0b-f5cf584204ac admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.628889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.629444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.629779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.630086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.630748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.634101 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.653172 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1219/2471] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:44.654007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.662535 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:44.667271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ed04c-6fab-42e5-8fc8-23f0a580646d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:44.683213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:44.685495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:44.685495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:44.689330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] PATH_INFO: `/v3/projects/f607b69991bb4b05a57179eec7023195/users/caee6781dc7145b4a38d3659f2b44195/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:44.714148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f607b69991bb4b05a57179eec7023195, user_id=caee6781dc7145b4a38d3659f2b44195, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:44.717055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:44.887153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1253/2472] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:44 2022] PUT /identity/v3/projects/f607b69991bb4b05a57179eec7023195/users/caee6781dc7145b4a38d3659f2b44195/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:44.891856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:44.905822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732f0e97-408d-487f-8684-07d17726c1bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.050618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb78bad9-7fea-4930-87d4-7cbaf880e07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.051198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78bad9-7fea-4930-87d4-7cbaf880e07d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.051509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78bad9-7fea-4930-87d4-7cbaf880e07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.051785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb78bad9-7fea-4930-87d4-7cbaf880e07d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.052479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78bad9-7fea-4930-87d4-7cbaf880e07d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.054162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb78bad9-7fea-4930-87d4-7cbaf880e07d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.065682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1220/2473] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:44 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:45.070439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.084632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788b098-9e4c-44d8-b4f9-a79e75cb0db7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.089447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.090285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.090739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.091042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] PATH_INFO: `/v3/projects/f607b69991bb4b05a57179eec7023195/users/caee6781dc7145b4a38d3659f2b44195/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.093293 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f607b69991bb4b05a57179eec7023195, user_id=caee6781dc7145b4a38d3659f2b44195, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.096213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.109009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1254/2474] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:45 2022] PUT /identity/v3/projects/f607b69991bb4b05a57179eec7023195/users/caee6781dc7145b4a38d3659f2b44195/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:45.121872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.122168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.122455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.175353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] MFA Rules not processed for user `caee6781dc7145b4a38d3659f2b44195`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:45.176741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.308820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1221/2475] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:45 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:45.321519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:45.323694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:45.396660 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0912cee3-1cc4-4fe0-b62f-f5b85f339652 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:45.400520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561969e8-72c2-4cfe-96dc-68a0f61ca494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:45.401250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561969e8-72c2-4cfe-96dc-68a0f61ca494 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:45.401693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561969e8-72c2-4cfe-96dc-68a0f61ca494 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:45.402092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561969e8-72c2-4cfe-96dc-68a0f61ca494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:45.413620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561969e8-72c2-4cfe-96dc-68a0f61ca494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:45.415486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561969e8-72c2-4cfe-96dc-68a0f61ca494 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:45.427502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1255/2476] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:46.935026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:46.985373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:47.027838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c7041-e83b-4437-83bc-a2c22a452c66 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:47.032607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:47.033686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.034227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.034515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] PATH_INFO: `/v3/users/e682fbdcbec14ea3bbdc02d73d1b981c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.036351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e682fbdcbec14ea3bbdc02d73d1b981c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:47.038714 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:47.264913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e682fbdcbec14ea3bbdc02d73d1b981c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.272223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e682fbdcbec14ea3bbdc02d73d1b981c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.344249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e682fbdcbec14ea3bbdc02d73d1b981c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.344846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e682fbdcbec14ea3bbdc02d73d1b981c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.359528 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e682fbdcbec14ea3bbdc02d73d1b981c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:47.368902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1222/2477] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:46 2022] DELETE /identity/v3/users/e682fbdcbec14ea3bbdc02d73d1b981c => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:47.373494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.374621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.374738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.398024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:47.399327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:47.491224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1256/2478] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:47.502347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:47.504238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:47.594122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8957d3-5144-4a42-bf7e-4d030b84dfad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:47.601075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:47.601778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:47.602069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:47.602353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:47.614290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:47.615747 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:47.647117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1223/2479] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:48.354149 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:48.355603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:48.380478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2ec816-ddac-42ca-b006-faedb61ec5e2 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:48.393226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:48.393925 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:48.394434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:48.394874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] PATH_INFO: `/v3/projects/630a924648ac4094a731c69c96212e0c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:48.397617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=630a924648ac4094a731c69c96212e0c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:48.399869 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:48.569266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '630a924648ac4094a731c69c96212e0c', 'request_id': 'req-222578a3-636d-4df1-9c68-16002c3c2c0b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.681181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '630a924648ac4094a731c69c96212e0c', 'request_id': 'req-222578a3-636d-4df1-9c68-16002c3c2c0b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.688756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '630a924648ac4094a731c69c96212e0c', 'request_id': 'req-222578a3-636d-4df1-9c68-16002c3c2c0b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.694842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:48.703061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] The token cache is being invalidate because project 630a924648ac4094a731c69c96212e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:48.716955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.716955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:48.716955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1257/2480] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:48 2022] DELETE /identity/v3/projects/630a924648ac4094a731c69c96212e0c => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:48.718102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:48.728947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d1efd-30aa-4f63-abaa-d1225ca32264 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:48.798549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:48.806698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:48.807616 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:48.807911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:48.808294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] PATH_INFO: `/v3/users/caee6781dc7145b4a38d3659f2b44195` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:48.809892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caee6781dc7145b4a38d3659f2b44195)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:48.813353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:49.096813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caee6781dc7145b4a38d3659f2b44195'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.105393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caee6781dc7145b4a38d3659f2b44195'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.218887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caee6781dc7145b4a38d3659f2b44195'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.219530 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caee6781dc7145b4a38d3659f2b44195'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.235745 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caee6781dc7145b4a38d3659f2b44195'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:49.244206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1224/2481] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:48 2022] DELETE /identity/v3/users/caee6781dc7145b4a38d3659f2b44195 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:49.788010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:49.789803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:49.804146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222578a3-636d-4df1-9c68-16002c3c2c0b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:49.877464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:49.877846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:49.878223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:49.878430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] PATH_INFO: `/v3/users/ef39008f78bd4c6f9547b35de6738c77` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:49.879723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef39008f78bd4c6f9547b35de6738c77)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:49.881781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:49.888026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:49.890181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:49.910567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46737afd-a74e-4124-b839-aa8bf92fd9b2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:49.917661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:49.918406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:49.918834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:49.919260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] PATH_INFO: `/v3/projects/f607b69991bb4b05a57179eec7023195` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:49.921481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f607b69991bb4b05a57179eec7023195)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:49.924748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.094715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef39008f78bd4c6f9547b35de6738c77'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.110413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef39008f78bd4c6f9547b35de6738c77'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.111566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef39008f78bd4c6f9547b35de6738c77'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.112224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef39008f78bd4c6f9547b35de6738c77'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.117700 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f607b69991bb4b05a57179eec7023195', 'request_id': 'req-27f05bfd-26c2-4696-bb12-ec54c106a8bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.126682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d2eb0a8d-6d78-43b2-b686-d303d0740b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef39008f78bd4c6f9547b35de6738c77'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.129237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f607b69991bb4b05a57179eec7023195', 'request_id': 'req-27f05bfd-26c2-4696-bb12-ec54c106a8bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.137177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f607b69991bb4b05a57179eec7023195', 'request_id': 'req-27f05bfd-26c2-4696-bb12-ec54c106a8bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.205817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1258/2482] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:17:49 2022] DELETE /identity/v3/users/ef39008f78bd4c6f9547b35de6738c77 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:50.210732 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.211213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.212396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.235879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:50.237172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.305217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1259/2483] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:50.308511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] The token cache is being invalidate because project f607b69991bb4b05a57179eec7023195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:50.308828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.309812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-27f05bfd-26c2-4696-bb12-ec54c106a8bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:50.311735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1225/2484] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:49 2022] DELETE /identity/v3/projects/f607b69991bb4b05a57179eec7023195 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:50.315160 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.316646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.317291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c31492-076d-43e5-ae44-853ab230ffca None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.317583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c31492-076d-43e5-ae44-853ab230ffca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.317941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c31492-076d-43e5-ae44-853ab230ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.331627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:50.340231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-45c31492-076d-43e5-ae44-853ab230ffca None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:50.341487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-45c31492-076d-43e5-ae44-853ab230ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.351006 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1226/2485] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:50.361139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.362012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.362012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.381515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c3c82-375e-4e54-a491-61d78e6e32df None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.381790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:50.382999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.385310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.385971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.386953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.386953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.388917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.391368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1227/2486] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:50.395574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.402153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.403623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.405662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b30c9-b0bb-4cd0-89d7-cffc009ed4fa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.410872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.411836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.415693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.415693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.415693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:50.415693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.416235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1260/2487] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:50.416334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.432607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1228/2488] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:17:50.433894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.449230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87098441-43d9-4aa1-a017-e5465c7adb04 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.454825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.455444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.455938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.456079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.456624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.458687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.597982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1261/2489] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/projects => generated 409 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:50.602346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.612995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec75a6f8-231b-4b81-b288-6b82fb9828f5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.645318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:50.680943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.681310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.681535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.681856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.682595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.684532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.795776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1229/2490] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:50 2022] POST /identity/v3/users => generated 305 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:50.798682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.809165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0712b4e6-9bcb-49f3-9ba0-c289ad2f6456 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.814632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.815303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.815838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.815838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.816456 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.818783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.832695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1262/2491] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:50.836927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.849199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49845967-03fc-4f0f-88c6-b0cd17af8829 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.855513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.856222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.856597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.856938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.857657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.859637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.873243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1230/2492] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:50.875132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.881038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:50.884780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:50.892755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f300358-ba19-40c6-8b52-2c9974e7e6f3 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.899971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a951e2a0-8304-417d-a4bc-7d63ff334a44 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:50.904818 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.906383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.907177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.907729 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0f907a-faf5-43bf-86c7-3a330b0c2118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:50.909059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f907a-faf5-43bf-86c7-3a330b0c2118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:50.909059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f907a-faf5-43bf-86c7-3a330b0c2118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:50.909271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0f907a-faf5-43bf-86c7-3a330b0c2118 admin admin] PATH_INFO: `/v3/projects/4014d3262a5f4fb8a74957f84b963e6f/users/4065338eb83b4d12a598512c163247ab/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.910120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] PATH_INFO: `/v3/projects/4cb7ea861caf4e0e9047dab83da411a4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:50.916264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb7ea861caf4e0e9047dab83da411a4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.917768 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:50.927469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f907a-faf5-43bf-86c7-3a330b0c2118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4014d3262a5f4fb8a74957f84b963e6f, user_id=4065338eb83b4d12a598512c163247ab, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:50.942144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0f907a-faf5-43bf-86c7-3a330b0c2118 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.053034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1263/2493] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:50 2022] PUT /identity/v3/projects/4014d3262a5f4fb8a74957f84b963e6f/users/4065338eb83b4d12a598512c163247ab/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:51.062828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.063123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.063393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.079203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb7ea861caf4e0e9047dab83da411a4', 'request_id': 'req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.089568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb7ea861caf4e0e9047dab83da411a4', 'request_id': 'req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.098338 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb7ea861caf4e0e9047dab83da411a4', 'request_id': 'req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.112928 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] MFA Rules not processed for user `4065338eb83b4d12a598512c163247ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.114294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.170788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] The token cache is being invalidate because project 4cb7ea861caf4e0e9047dab83da411a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:17:51.171316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.172789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d5eb6ee8-8628-49c6-afdd-8fbcc47ec500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:17:51.174455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1231/2494] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:17:50 2022] DELETE /identity/v3/projects/4cb7ea861caf4e0e9047dab83da411a4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:51.181499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8e2c5-aef8-4416-93f4-d65f5a48653f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.181857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8e2c5-aef8-4416-93f4-d65f5a48653f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.182227 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8e2c5-aef8-4416-93f4-d65f5a48653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.182892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1264/2495] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:51.187347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.210815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-42a8e2c5-aef8-4416-93f4-d65f5a48653f None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.212352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-42a8e2c5-aef8-4416-93f4-d65f5a48653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.241636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:51.258937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297bed9-e697-4461-8c53-7ec97c333eac None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.264054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.266047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.266359 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.266624 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.267283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.269285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.282943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1232/2496] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:51.291222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.291505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.291788 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.313124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.314378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.322541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1233/2497] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:51.326489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.337436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ee030-cbf1-43fc-b0aa-020b7f584bca None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.342215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.343710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.344048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.344333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.344979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:17:51.345292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.346928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.361669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1234/2498] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:17:51.364178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.374349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23917cc-c646-49cf-859d-f3939f843c5f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.379223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.379824 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.380106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.380383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.381015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.382939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.410232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1265/2499] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/projects => generated 409 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:51.414428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.416207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.484131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2686a1f-57bf-4481-8e12-54cf61351bdb admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.488923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.489543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.489858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.490138 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.490776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.492705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.504672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:51.528827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1235/2500] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/projects => generated 371 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:51.529719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.596749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc0fb12-4982-41c7-9911-3d865928af01 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.601954 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.602575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.602858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.603137 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.603877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.605801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.652493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1266/2501] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/users => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:17:51.656812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.667226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d2dc45-5208-4671-b25a-2a4d16ab9532 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.672828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.673451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.673758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.674046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.674684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.676286 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.687109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1267/2502] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:51.690769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.700518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b593f-8a22-475c-92ac-0a3954d71361 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.706120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.706767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.707053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.708240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] PATH_INFO: `/v3/projects/7ec2d08e21b246a89c79ef1a1d77eda5/users/5fde027e00f94e68b80350adb9bcbdc5/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.722496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d08e21b246a89c79ef1a1d77eda5, user_id=5fde027e00f94e68b80350adb9bcbdc5, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.725264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.744927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1236/2503] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:51.749243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.759265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc18eb2-6244-4567-ac86-0a66c5bcefd6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.763986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.765122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.765406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.765681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.766347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.767930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.778113 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1237/2504] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:51.782651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.792792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8f4d2-f9b3-4d4b-9761-eca128a25fb9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.797579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.798215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.798503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.798782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.799415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.801012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.812318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1238/2505] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:51.816594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.821678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1268/2506] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:51 2022] PUT /identity/v3/projects/7ec2d08e21b246a89c79ef1a1d77eda5/users/5fde027e00f94e68b80350adb9bcbdc5/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:51.825969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.827722 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6183090-4c97-49ef-82e2-05dc3e3dffd9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.888500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c88f32-3ca8-4d44-86a8-b764411839b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.889177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c88f32-3ca8-4d44-86a8-b764411839b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.890253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c88f32-3ca8-4d44-86a8-b764411839b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.890696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c88f32-3ca8-4d44-86a8-b764411839b9 admin admin] PATH_INFO: `/v3/projects/ba3bd03ea50c4553b9b98b61a4d835dd/users/9352fbf7d98b40d7954ce1ff3d968761/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.892465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddc2982-c94e-4c77-95a1-c2125df79759 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.897610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.898245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.898526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.898802 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.899440 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.901094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.906098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c88f32-3ca8-4d44-86a8-b764411839b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3bd03ea50c4553b9b98b61a4d835dd, user_id=9352fbf7d98b40d7954ce1ff3d968761, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.908800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c88f32-3ca8-4d44-86a8-b764411839b9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.912288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1269/2507] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:51 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:17:51.916815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:51.927898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca72fdd-e1e9-4799-83cf-85406c417165 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:51.933530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e044e28b-a455-47ec-b9da-2a29eb07b439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:51.934226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e28b-a455-47ec-b9da-2a29eb07b439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.934508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e28b-a455-47ec-b9da-2a29eb07b439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.934787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e044e28b-a455-47ec-b9da-2a29eb07b439 admin admin] PATH_INFO: `/v3/projects/7ec2d08e21b246a89c79ef1a1d77eda5/users/5fde027e00f94e68b80350adb9bcbdc5/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.937140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044e28b-a455-47ec-b9da-2a29eb07b439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec2d08e21b246a89c79ef1a1d77eda5, user_id=5fde027e00f94e68b80350adb9bcbdc5, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:51.939848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e044e28b-a455-47ec-b9da-2a29eb07b439 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:51.951198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1270/2508] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:51 2022] PUT /identity/v3/projects/7ec2d08e21b246a89c79ef1a1d77eda5/users/5fde027e00f94e68b80350adb9bcbdc5/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:51.961938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.962262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.962542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:51.993521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1239/2509] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:51 2022] PUT /identity/v3/projects/ba3bd03ea50c4553b9b98b61a4d835dd/users/9352fbf7d98b40d7954ce1ff3d968761/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:17:51.993845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] MFA Rules not processed for user `5fde027e00f94e68b80350adb9bcbdc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:51.997539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:51.997754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:51.997754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:51.997754 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.017774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:52.018995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:52.043288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:52.087501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1240/2510] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:52.091118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1271/2511] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:51 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:17:52.099948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.101355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:52.106763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.109473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:52.193438 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d62596-a967-47a1-8f53-155a8795a1af None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.197503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.197997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d50f6b-27f5-41b9-a150-b0a616e08556 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.198183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.198442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.198946 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.208200 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af4bf7-1ae8-4f03-82bd-297cf8b0472c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.208815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af4bf7-1ae8-4f03-82bd-297cf8b0472c service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.209097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af4bf7-1ae8-4f03-82bd-297cf8b0472c service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.209370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af4bf7-1ae8-4f03-82bd-297cf8b0472c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.215999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.218468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.230045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af4bf7-1ae8-4f03-82bd-297cf8b0472c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.230045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af4bf7-1ae8-4f03-82bd-297cf8b0472c service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.231075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1241/2512] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:52.234324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1272/2513] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:52 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:17:52.338883 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:52.635574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:52.645841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09109b6-be28-49b6-b5a3-dd8f54103e02 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:52.649445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:52.650091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:52.653950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:52.653950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:52.653950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:52.714411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:52.715834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:52.725014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1242/2514] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:17:52 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:17:54.100078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7000-02ac-4875-9520-a57d6eda5966 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:54.103003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7000-02ac-4875-9520-a57d6eda5966 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:54.103003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520b7000-02ac-4875-9520-a57d6eda5966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:54.139863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-520b7000-02ac-4875-9520-a57d6eda5966 None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:54.141162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-520b7000-02ac-4875-9520-a57d6eda5966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:54.208378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1273/2515] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 30 01:17:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:54.224073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:54.226047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:54.311051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f148dcd-dfe0-4c8d-b56b-3dd0b1229ad8 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:54.317555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:54.318200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:54.318598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:54.319375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:54.332934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:54.332934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:54.339147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1243/2516] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:17:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 01:17:59.077309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.077999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.078302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.114266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] MFA Rules not processed for user `9352fbf7d98b40d7954ce1ff3d968761`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:17:59.115905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:17:59.158173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:59.193479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1274/2517] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:17:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:17:59.197917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.210091 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af2998-5eb3-4860-bbe1-4a42d1c93fd2 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.215110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.215726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.217992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.217992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.217992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.219431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.236124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:17:59.317847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1244/2518] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:17:59 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:59.319133 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.332570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199914ab-56b8-4565-b566-a1248c053ebb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.398808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.404142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.404142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.404142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.404142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.404142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.522157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1275/2519] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:17:59 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:17:59.526223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.537010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1601c39-9896-441e-a565-45060b61f4bf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.542935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.543566 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.543851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.544127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.544778 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.547022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.558368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1245/2520] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:59.564388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.576004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26346b96-8b40-4600-9233-95d16b808839 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.581332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.582067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.582347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.582631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] PATH_INFO: `/v3/projects/0b43e37e5945406ca905cecad78e76c4/users/c3b49ba2b14545acbb9cc7cd9511a2aa/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.600666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b43e37e5945406ca905cecad78e76c4, user_id=c3b49ba2b14545acbb9cc7cd9511a2aa, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.603507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.721524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1276/2521] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:59 2022] PUT /identity/v3/projects/0b43e37e5945406ca905cecad78e76c4/users/c3b49ba2b14545acbb9cc7cd9511a2aa/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:17:59.725450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.736687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1850fd-0eb3-45d6-9d22-dfe686033a9c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.825149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c4161-81dc-44c6-8844-e74606348d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.825790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4161-81dc-44c6-8844-e74606348d8b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.826078 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4161-81dc-44c6-8844-e74606348d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.826355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c4161-81dc-44c6-8844-e74606348d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.827297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c4161-81dc-44c6-8844-e74606348d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.829062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c4161-81dc-44c6-8844-e74606348d8b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.842892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1246/2522] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:17:59.844646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:17:59.856894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd64d94c-b7ed-41a6-af9f-acdbe4a84f1f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:17:59.863230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:17:59.864032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:17:59.864398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:17:59.864750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] PATH_INFO: `/v3/projects/0b43e37e5945406ca905cecad78e76c4/users/c3b49ba2b14545acbb9cc7cd9511a2aa/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:17:59.866863 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b43e37e5945406ca905cecad78e76c4, user_id=c3b49ba2b14545acbb9cc7cd9511a2aa, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:17:59.869649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:17:59.880971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1277/2523] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:17:59 2022] PUT /identity/v3/projects/0b43e37e5945406ca905cecad78e76c4/users/c3b49ba2b14545acbb9cc7cd9511a2aa/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:01.037161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.037651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.037851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.058447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] MFA Rules not processed for user `3002491f2c61493890d2e4272e9ba522`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:01.059650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:01.128848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1247/2524] 10.222.0.52 () {58 vars in 967 bytes} [Tue Aug 30 01:18:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:01.141340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:01.212360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2aada5-62c7-404a-9f77-f704b6a58293 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:01.217765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b4373-d2b6-4459-8ca8-558d511435d7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6a7f47fa1a8e4902a1a68219366274c2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:01.218486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4373-d2b6-4459-8ca8-558d511435d7 service swift] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:01.218771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4373-d2b6-4459-8ca8-558d511435d7 service swift] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:01.219044 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b4373-d2b6-4459-8ca8-558d511435d7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:01.227642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b4373-d2b6-4459-8ca8-558d511435d7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:01.229247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564b4373-d2b6-4459-8ca8-558d511435d7 service swift] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:01.238331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1278/2525] 10.222.0.52 () {60 vars in 1384 bytes} [Tue Aug 30 01:18:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 01:18:03.909046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.052125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d6222-fc92-4986-906d-11faefd2b4ec None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.058983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.059657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.060038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.060210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] PATH_INFO: `/v3/users/4065338eb83b4d12a598512c163247ab` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.066046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=4065338eb83b4d12a598512c163247ab)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.068174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.130649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:04.383961 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4065338eb83b4d12a598512c163247ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:04.414206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4065338eb83b4d12a598512c163247ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:04.519559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4065338eb83b4d12a598512c163247ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:04.520341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4065338eb83b4d12a598512c163247ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:04.535991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4065338eb83b4d12a598512c163247ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:04.573419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1248/2526] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:03 2022] DELETE /identity/v3/users/4065338eb83b4d12a598512c163247ab => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:04.574741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.575217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.575479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.618142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:04.622459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.647611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:04.776516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1279/2527] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:04.784245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:04.786775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:04.879307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea5a423-24ea-4052-94ea-144719cde5ec admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:04.884703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:04.886238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:04.886799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:04.887295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:04.901410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:04.906259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:04.920923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1249/2528] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:05.804381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:05.807270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:05.829173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6973eb-0564-48e6-8911-789a29f9f111 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:05.862668 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:05.863425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:05.863706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:05.863980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] PATH_INFO: `/v3/projects/4014d3262a5f4fb8a74957f84b963e6f` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:05.865447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4014d3262a5f4fb8a74957f84b963e6f)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:05.867589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:06.049539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4014d3262a5f4fb8a74957f84b963e6f', 'request_id': 'req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.158755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4014d3262a5f4fb8a74957f84b963e6f', 'request_id': 'req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.167857 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4014d3262a5f4fb8a74957f84b963e6f', 'request_id': 'req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.211892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] The token cache is being invalidate because project 4014d3262a5f4fb8a74957f84b963e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:06.212908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.213650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.219140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1280/2529] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:05 2022] DELETE /identity/v3/projects/4014d3262a5f4fb8a74957f84b963e6f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:06.221126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:06.255563 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1978d654-4ff0-4dec-bc3c-3c367135b7cc service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:06.403372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:06.404389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:06.409445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:06.409445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] PATH_INFO: `/v3/users/5fde027e00f94e68b80350adb9bcbdc5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:06.409445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fde027e00f94e68b80350adb9bcbdc5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:06.410564 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:06.634485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fde027e00f94e68b80350adb9bcbdc5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.650900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:06.659083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fde027e00f94e68b80350adb9bcbdc5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.751561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fde027e00f94e68b80350adb9bcbdc5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.753522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fde027e00f94e68b80350adb9bcbdc5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.770429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8cc47fbd-38f8-42e4-b0e0-8bc2a0c2d2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fde027e00f94e68b80350adb9bcbdc5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:06.780248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1250/2530] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:06 2022] DELETE /identity/v3/users/5fde027e00f94e68b80350adb9bcbdc5 => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:07.359194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:07.360924 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:07.401090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff1ad7e-6fdc-4759-a4a4-f03fd557d14a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:07.489116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:07.490242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:07.490999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:07.491510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] PATH_INFO: `/v3/projects/7ec2d08e21b246a89c79ef1a1d77eda5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:07.492774 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec2d08e21b246a89c79ef1a1d77eda5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:07.494796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:07.703351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:07.711678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec2d08e21b246a89c79ef1a1d77eda5', 'request_id': 'req-d39f76a5-a357-42c9-8547-2127c49b4de8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.901240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec2d08e21b246a89c79ef1a1d77eda5', 'request_id': 'req-d39f76a5-a357-42c9-8547-2127c49b4de8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.908483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec2d08e21b246a89c79ef1a1d77eda5', 'request_id': 'req-d39f76a5-a357-42c9-8547-2127c49b4de8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.920498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] The token cache is being invalidate because project 7ec2d08e21b246a89c79ef1a1d77eda5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:07.921028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.921864 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d39f76a5-a357-42c9-8547-2127c49b4de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:07.923344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1281/2531] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:07 2022] DELETE /identity/v3/projects/7ec2d08e21b246a89c79ef1a1d77eda5 => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:07.928497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557adfe-f2c5-4b42-a621-4b24f44911c5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:07.928999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557adfe-f2c5-4b42-a621-4b24f44911c5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:07.929108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e557adfe-f2c5-4b42-a621-4b24f44911c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:07.951804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e557adfe-f2c5-4b42-a621-4b24f44911c5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:07.953341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e557adfe-f2c5-4b42-a621-4b24f44911c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:07.956148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:07.956646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:07.957075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:07.993466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] MFA Rules not processed for user `c3b49ba2b14545acbb9cc7cd9511a2aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:07.995571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.028179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1251/2532] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:08.037127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.037976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.038697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.069009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:08.070291 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.079201 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1252/2533] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:08.083628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.095019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786f73-ca90-40f4-a65b-eab84b01983c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.097400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1282/2534] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:07 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:08.103593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.104301 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.104735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.105043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.105861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:08.106241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.108830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.111914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.113341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.121849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1253/2535] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:18:08.134012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.144375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ffc5b-0afc-4d8f-a1b5-b68e260c0cb4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.149248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.149873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.150156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.150428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.151116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.153015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.190002 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597da95-7b4c-49cb-9735-9d4a03512715 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.195910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.197473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.197772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.198053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.201493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:08.246407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1254/2536] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:08.247434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.302183 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.303730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.318332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1283/2537] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:08.321643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:08.396532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49567e87-7e6c-40f5-a4da-114b85a9f4ce admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.413275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.414551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.415002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.415473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.418009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.423228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.564176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1255/2538] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:08.570147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.586960 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c1f2f1-ec55-4206-bdbf-d819cf8decd0 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.607876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.607876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.607876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.607876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.608368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.608368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.625583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1284/2539] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:08.629381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.653498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ca12ca-a7d3-4547-b7cb-64f3d9308349 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.659770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf8d2e-33ef-424d-9947-b1bfa79f6371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.660514 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf8d2e-33ef-424d-9947-b1bfa79f6371 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.660903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf8d2e-33ef-424d-9947-b1bfa79f6371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.661287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf8d2e-33ef-424d-9947-b1bfa79f6371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.669231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf8d2e-33ef-424d-9947-b1bfa79f6371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.672581 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf8d2e-33ef-424d-9947-b1bfa79f6371 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.693007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:08.705842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1256/2540] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:08 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:08.711065 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:08.722191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73470594-32e9-49e9-9c10-d0d4e5da4f0f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:08.726975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:08.727881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.728396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.728837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] PATH_INFO: `/v3/projects/bc5b49ad56324f7d8daca54a59b0358c/users/b857bf5bd09f40028015b348782ab297/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.746006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5b49ad56324f7d8daca54a59b0358c, user_id=b857bf5bd09f40028015b348782ab297, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:08.748908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:08.867228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1285/2541] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:08 2022] PUT /identity/v3/projects/bc5b49ad56324f7d8daca54a59b0358c/users/b857bf5bd09f40028015b348782ab297/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:08.869699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b46a-767e-4fcb-94e8-cb8775f7ee12 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:08.870016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b46a-767e-4fcb-94e8-cb8775f7ee12 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:08.870401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593b46a-767e-4fcb-94e8-cb8775f7ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:08.915342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4593b46a-767e-4fcb-94e8-cb8775f7ee12 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:08.916589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4593b46a-767e-4fcb-94e8-cb8775f7ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.004702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1257/2542] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:09.015321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:09.017472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:09.105979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217b40d-0a26-4531-a1e2-e900415cb036 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:09.110016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:09.110637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:09.110923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:09.111202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:09.120767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:09.122264 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:09.133679 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1286/2543] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:11.283086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.283540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.283669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.314628 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] MFA Rules not processed for user `b857bf5bd09f40028015b348782ab297`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:11.315864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:11.399131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1258/2544] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:11.403775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.419741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee7e1f-b2d3-4545-bb6f-0b0b2e488f26 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.426813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.427836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.428813 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.429251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.430897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.433071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.577715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1287/2545] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:11 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:11.578941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.594872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404da95-a27e-48f5-9c77-5f6ddbad0466 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.688828 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.693239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.693239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.693239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.693239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.693239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.847733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.849534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1259/2546] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:11 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:11.861310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0104fb7-9082-4bd9-a6ed-960eb7d01db6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.867271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.871591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.871591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.871591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.871591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.872391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.915082 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:11.916142 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1288/2547] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:11.929248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c14beaf-5be9-4ec4-8880-4af3e8afff68 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:11.937887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:11.939555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:11.940055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:11.944203 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] PATH_INFO: `/v3/projects/818a0c6fa4a44406806a97d6de460c94/users/4a0c2a52f0dd4cb0996d3f2733b74c4c/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:11.960364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=818a0c6fa4a44406806a97d6de460c94, user_id=4a0c2a52f0dd4cb0996d3f2733b74c4c, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:11.963955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:11.969004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:12.087353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1260/2548] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:11 2022] PUT /identity/v3/projects/818a0c6fa4a44406806a97d6de460c94/users/4a0c2a52f0dd4cb0996d3f2733b74c4c/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:12.091518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:12.104337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69401c00-ba8b-43d1-93fb-c30e3ceae369 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:12.184901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b6a2d-fc49-4e01-8e4d-4f1211c38eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:12.185770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6a2d-fc49-4e01-8e4d-4f1211c38eec admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.186212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6a2d-fc49-4e01-8e4d-4f1211c38eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.186493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6a2d-fc49-4e01-8e4d-4f1211c38eec admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.187165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b6a2d-fc49-4e01-8e4d-4f1211c38eec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:12.188874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b6a2d-fc49-4e01-8e4d-4f1211c38eec admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:12.201282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1289/2549] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:12.205887 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:12.219349 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee56c29c-f557-451d-a1e5-13f385b99cb9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:12.225994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7302cb80-a735-4e1a-9db2-85d335cc4964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:12.226993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302cb80-a735-4e1a-9db2-85d335cc4964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:12.227618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302cb80-a735-4e1a-9db2-85d335cc4964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:12.228014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7302cb80-a735-4e1a-9db2-85d335cc4964 admin admin] PATH_INFO: `/v3/projects/818a0c6fa4a44406806a97d6de460c94/users/4a0c2a52f0dd4cb0996d3f2733b74c4c/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:12.230803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302cb80-a735-4e1a-9db2-85d335cc4964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=818a0c6fa4a44406806a97d6de460c94, user_id=4a0c2a52f0dd4cb0996d3f2733b74c4c, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:12.239092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7302cb80-a735-4e1a-9db2-85d335cc4964 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:12.269236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1261/2550] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:12 2022] PUT /identity/v3/projects/818a0c6fa4a44406806a97d6de460c94/users/4a0c2a52f0dd4cb0996d3f2733b74c4c/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:13.014040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:13.015259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:13.016438 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:13.037495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] MFA Rules not processed for user `9352fbf7d98b40d7954ce1ff3d968761`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:13.039170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:13.067845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:13.116613 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1290/2551] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:14.160183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:14.160725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:14.161326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:14.243552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] MFA Rules not processed for user `4a0c2a52f0dd4cb0996d3f2733b74c4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:14.245033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:14.505525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1262/2552] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:14.512698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:14.514886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:14.648194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dde0c4-196b-4e1a-8e34-968670796882 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:14.652657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:14.653594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:14.654395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:14.663048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:14.764416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:14.766155 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:14.778336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1291/2553] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:15.846720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:15.848190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:15.955973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43864d94-3cbc-4d2c-bc1f-3afa1a21cb8d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:15.961607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:15.962451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:15.962806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:15.963153 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] PATH_INFO: `/v3/users/fdf94212f7654ff396be6dc2c2ca4874` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:15.970219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdf94212f7654ff396be6dc2c2ca4874)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:15.973202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:16.200503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdf94212f7654ff396be6dc2c2ca4874'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.208128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdf94212f7654ff396be6dc2c2ca4874'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.291737 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdf94212f7654ff396be6dc2c2ca4874'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.292360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdf94212f7654ff396be6dc2c2ca4874'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.299099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:16.306696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdf94212f7654ff396be6dc2c2ca4874'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.316340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1263/2554] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:15 2022] DELETE /identity/v3/users/fdf94212f7654ff396be6dc2c2ca4874 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:16.562715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.574313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b6a1f-7b0c-4490-b8a6-0f1e68cbde0f service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:16.637280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:16.637866 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.638150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.638428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] PATH_INFO: `/v3/projects/c6f3997439524d4583ece79273a55da4` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.640256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f3997439524d4583ece79273a55da4)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:16.642464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:16.673512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:16.765970 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f3997439524d4583ece79273a55da4', 'request_id': 'req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.834381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f3997439524d4583ece79273a55da4', 'request_id': 'req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.842278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f3997439524d4583ece79273a55da4', 'request_id': 'req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.853586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] The token cache is being invalidate because project c6f3997439524d4583ece79273a55da4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:16.853919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.854584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:16.856174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1292/2555] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:16 2022] DELETE /identity/v3/projects/c6f3997439524d4583ece79273a55da4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:16.862366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.864040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:16.878619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d9b06-e432-4d7e-8d80-0e4f5ff57b28 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:16.952165 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:16.954126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:16.954126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:16.954126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] PATH_INFO: `/v3/users/87e615b6b71f46d0b4d6c1867b6b58a2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:16.958404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e615b6b71f46d0b4d6c1867b6b58a2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:16.958404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:16.967682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:16.969425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.032680 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d9ec4-8448-40df-ac8d-6d6eb829cf2d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.036382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.036995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.037279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.037556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.039712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.109914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.111520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.124074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1293/2556] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:17.283027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e615b6b71f46d0b4d6c1867b6b58a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.294539 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e615b6b71f46d0b4d6c1867b6b58a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.415727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e615b6b71f46d0b4d6c1867b6b58a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.416571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e615b6b71f46d0b4d6c1867b6b58a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.435417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e615b6b71f46d0b4d6c1867b6b58a2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:17.445149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1264/2557] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:16 2022] DELETE /identity/v3/users/87e615b6b71f46d0b4d6c1867b6b58a2 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:17.808560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:17.810067 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:17.823982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb2dfe-92f2-41e9-b362-c6a131b49331 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:17.895697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:17.896340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:17.900355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:17.900355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] PATH_INFO: `/v3/projects/66adbbaaffa64f48aa2dc9417803c9bb` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:17.900355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=66adbbaaffa64f48aa2dc9417803c9bb)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:17.900355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:17.902780 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:18.086035 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66adbbaaffa64f48aa2dc9417803c9bb', 'request_id': 'req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.211926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66adbbaaffa64f48aa2dc9417803c9bb', 'request_id': 'req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.219673 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66adbbaaffa64f48aa2dc9417803c9bb', 'request_id': 'req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.231396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] The token cache is being invalidate because project 66adbbaaffa64f48aa2dc9417803c9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:18.231794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.232389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:18.237872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1294/2558] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:17 2022] DELETE /identity/v3/projects/66adbbaaffa64f48aa2dc9417803c9bb => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:22.400882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:22.402423 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:22.465220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e2dc77-871e-4301-8463-cfa143ff819c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:22.490852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:22.522136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:22.522759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:22.523038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:22.523315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:22.525541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:22.593450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:22.595007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:22.612669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1265/2559] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:26.283529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:26.285543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:26.385355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31019c51-dc7a-4a1a-b8fb-d96a668ccc0b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:26.390627 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:26.393541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:26.393541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:26.393541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:26.394185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:26.406713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:26.408702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:26.420302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1295/2560] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:26.426330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:26.427917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:26.450425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd787322-4c13-48b7-80d4-73aceed92094 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:26.457122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:26.457890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:26.458427 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:26.458888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] PATH_INFO: `/v3/users/942d175e7e2742728e66fea84c851038` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:26.460145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=942d175e7e2742728e66fea84c851038)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:26.462161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:26.683474 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '942d175e7e2742728e66fea84c851038'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.691848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '942d175e7e2742728e66fea84c851038'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.805537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '942d175e7e2742728e66fea84c851038'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.806285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '942d175e7e2742728e66fea84c851038'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.819974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-03e1cf9c-5f0b-46eb-8bcc-a4fd2b939d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '942d175e7e2742728e66fea84c851038'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:26.825919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1266/2561] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:26 2022] DELETE /identity/v3/users/942d175e7e2742728e66fea84c851038 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:27.114476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.126239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845c0cf1-6256-4a82-9901-93233367267f service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.221672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.221672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.221672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.221672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] PATH_INFO: `/v3/projects/11c8f48af7394a75ab6013ca066751a7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.222154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c8f48af7394a75ab6013ca066751a7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.222154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.248412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:27.413214 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c8f48af7394a75ab6013ca066751a7', 'request_id': 'req-0ca732df-3d56-430e-bbba-37d7704d920c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.563760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c8f48af7394a75ab6013ca066751a7', 'request_id': 'req-0ca732df-3d56-430e-bbba-37d7704d920c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.571620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c8f48af7394a75ab6013ca066751a7', 'request_id': 'req-0ca732df-3d56-430e-bbba-37d7704d920c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.582875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] The token cache is being invalidate because project 11c8f48af7394a75ab6013ca066751a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:27.583146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.583832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0ca732df-3d56-430e-bbba-37d7704d920c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:27.585305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1296/2562] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:27 2022] DELETE /identity/v3/projects/11c8f48af7394a75ab6013ca066751a7 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:27.590309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.590789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.590927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.611489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:27.612718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.664944 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:27.684000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1267/2563] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:27.693543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.693882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.694181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.714979 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:27.716202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:27.724956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1297/2564] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:27.735646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.745296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0426253-6a72-4d04-9159-0d00bbc06284 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.750039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.750640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.750921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.751196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.752355 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:27.752652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.754261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.765952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1268/2565] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:27.773969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.783855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5d448-4dc3-4577-85b7-2b2c6320c5d9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:27.789230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:27.789865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:27.790247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:27.790554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:27.791188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:27.793111 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:27.939745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1298/2566] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:27 2022] POST /identity/v3/projects => generated 403 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:27.944115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:27.954250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24505ad2-263c-40d2-a2b5-d4d58a8545dc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.011716 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.012124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.012402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.012675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.013295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.015251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.106986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1269/2567] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:27 2022] POST /identity/v3/users => generated 302 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:28.112074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.122621 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d939ed-cf8a-4cfe-8380-b40652054c98 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.128124 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.128724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.129005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.129314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.129985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.131592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.142102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1299/2568] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:28.146808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.157225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e089-4a27-41d4-ab49-a435c64604c4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.162509 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e764d-ee3d-4a46-a73c-145188361a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.163442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e764d-ee3d-4a46-a73c-145188361a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.163729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e764d-ee3d-4a46-a73c-145188361a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.164005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e764d-ee3d-4a46-a73c-145188361a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.164646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e764d-ee3d-4a46-a73c-145188361a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.166310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e764d-ee3d-4a46-a73c-145188361a21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.178174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1270/2569] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:28.182687 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.193640 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019bd8e-98f1-4bda-996c-53e46a29c0d7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.199342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.199516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.199737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.200179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] PATH_INFO: `/v3/projects/6bd6d7b75f524b87aa47bab31516ba5c/users/0f9644c816b749298346a335796c99dc/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.217867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd6d7b75f524b87aa47bab31516ba5c, user_id=0f9644c816b749298346a335796c99dc, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.220847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.332524 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1300/2570] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:28 2022] PUT /identity/v3/projects/6bd6d7b75f524b87aa47bab31516ba5c/users/0f9644c816b749298346a335796c99dc/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:28.338290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fbde-6a00-49c0-bd6d-2e955e9a53da None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.338290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fbde-6a00-49c0-bd6d-2e955e9a53da None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.338587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57fbde-6a00-49c0-bd6d-2e955e9a53da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.358914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9c57fbde-6a00-49c0-bd6d-2e955e9a53da None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:28.360138 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c57fbde-6a00-49c0-bd6d-2e955e9a53da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.422789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1271/2571] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:28.433599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:28.435475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:28.495931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7ea78d-4260-4f74-96d7-d787c7c0ac59 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:28.499853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:28.500472 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:28.500759 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:28.501072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:28.510859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:28.513066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:28.522650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1301/2572] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:30.001219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.001695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.001844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.034151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] MFA Rules not processed for user `0f9644c816b749298346a335796c99dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:30.035396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:30.084992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:30.136168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1272/2573] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:29 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:30.140719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.155093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6cec9-a276-4068-9479-59f7b34d1699 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.161176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.161845 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.162132 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.162404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.163390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.165876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.259216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1302/2574] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:30 2022] POST /identity/v3/projects => generated 405 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:30.263434 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.274966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c4af2-17fc-42d8-acdc-fac1df4b5798 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.351689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.352416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.352758 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.353679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.353936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.355918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.455045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1273/2575] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:30 2022] POST /identity/v3/users => generated 309 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:30.459466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.470364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc2670a-b4fa-43da-b62f-0d34223913ab admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.482413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.482413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.482413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.482413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.482858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.482858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.495050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1303/2576] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:30.496678 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.507351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c31ff0-9994-48bd-be6c-ba7c448818ee admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.512242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.514523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.514523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.514523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] PATH_INFO: `/v3/projects/95f3fc21abff496e842436d4fb4bcccf/users/7c2f297fc6e24542b6bbb9f883d8b271/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.529335 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f3fc21abff496e842436d4fb4bcccf, user_id=7c2f297fc6e24542b6bbb9f883d8b271, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.532025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.630179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1274/2577] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:30 2022] PUT /identity/v3/projects/95f3fc21abff496e842436d4fb4bcccf/users/7c2f297fc6e24542b6bbb9f883d8b271/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:30.631173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.641938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53313bb-797d-44c2-b81e-68eba09f81f8 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.655462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:30.705216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af72304-cbe6-48cf-a209-5c6ea63460f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.706150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af72304-cbe6-48cf-a209-5c6ea63460f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.706620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af72304-cbe6-48cf-a209-5c6ea63460f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.707066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af72304-cbe6-48cf-a209-5c6ea63460f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.707889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af72304-cbe6-48cf-a209-5c6ea63460f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.709694 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af72304-cbe6-48cf-a209-5c6ea63460f2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.725684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1304/2578] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:30.727031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:30.740186 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec464e-1e64-4568-a0a7-a334da8ac021 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:30.745231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:30.746074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:30.746383 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:30.746755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] PATH_INFO: `/v3/projects/95f3fc21abff496e842436d4fb4bcccf/users/7c2f297fc6e24542b6bbb9f883d8b271/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:30.748955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95f3fc21abff496e842436d4fb4bcccf, user_id=7c2f297fc6e24542b6bbb9f883d8b271, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:30.752155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:30.765551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1275/2579] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:30 2022] PUT /identity/v3/projects/95f3fc21abff496e842436d4fb4bcccf/users/7c2f297fc6e24542b6bbb9f883d8b271/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:32.363742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:32.364202 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:32.364297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:32.409339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] MFA Rules not processed for user `7c2f297fc6e24542b6bbb9f883d8b271`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:32.410792 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:32.535585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1305/2580] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:33.931776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:34.011038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec9ac8-040d-4180-a203-07c068d2d7c9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:34.015773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:34.017222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.018640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.019067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.021134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:34.037178 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:34.039979 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:34.054635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1276/2581] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:34.697859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:34.699358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:34.857123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd29548-15f1-4b9f-806a-f3ea5c6b678a None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:34.858344 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:34.859654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:34.860117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:34.860682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] PATH_INFO: `/v3/users/b857bf5bd09f40028015b348782ab297` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:34.863373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b857bf5bd09f40028015b348782ab297)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:34.865949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:35.084620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:35.090937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b857bf5bd09f40028015b348782ab297'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.103922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b857bf5bd09f40028015b348782ab297'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.104677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b857bf5bd09f40028015b348782ab297'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.105372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b857bf5bd09f40028015b348782ab297'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.119546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b857bf5bd09f40028015b348782ab297'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.210258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1306/2582] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:34 2022] DELETE /identity/v3/users/b857bf5bd09f40028015b348782ab297 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:35.224333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:35.239886 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87350925-7a19-4f29-89f0-8e2feb7ff387 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:35.278331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:35.306354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:35.307008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:35.307294 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:35.307568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] PATH_INFO: `/v3/users/0f9644c816b749298346a335796c99dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:35.309120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f9644c816b749298346a335796c99dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:35.311259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:35.512426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:35.514770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:35.551578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f9644c816b749298346a335796c99dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.557139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554e529-276b-47c3-bae0-c3d7f66de8ef admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:35.559859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f9644c816b749298346a335796c99dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.563810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:35.565289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:35.565791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:35.566270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] PATH_INFO: `/v3/projects/bc5b49ad56324f7d8daca54a59b0358c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:35.567803 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5b49ad56324f7d8daca54a59b0358c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:35.569901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:35.637630 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f9644c816b749298346a335796c99dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.638672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f9644c816b749298346a335796c99dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.654319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f9644c816b749298346a335796c99dc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.661931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1277/2583] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:35 2022] DELETE /identity/v3/users/0f9644c816b749298346a335796c99dc => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:35.706145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5b49ad56324f7d8daca54a59b0358c', 'request_id': 'req-444e9031-d7ef-4a2c-bdd9-f5836f84c495'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.788748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5b49ad56324f7d8daca54a59b0358c', 'request_id': 'req-444e9031-d7ef-4a2c-bdd9-f5836f84c495'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.798118 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5b49ad56324f7d8daca54a59b0358c', 'request_id': 'req-444e9031-d7ef-4a2c-bdd9-f5836f84c495'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.809095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] The token cache is being invalidate because project bc5b49ad56324f7d8daca54a59b0358c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:35.809505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.810285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:35.814905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1307/2584] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:35 2022] DELETE /identity/v3/projects/bc5b49ad56324f7d8daca54a59b0358c => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:35.816130 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:35.818161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:35.828796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:35.835011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:35.940155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d65ce3-3716-4261-918a-809de02a0ece admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:35.953452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e9031-d7ef-4a2c-bdd9-f5836f84c495 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:35.960368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:35.961219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:35.961358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:35.961610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:35.964107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:35.982467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:35.984089 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:36.001409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1308/2585] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:36.002386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:36.004763 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:36.060718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:36.061489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:36.061924 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:36.062478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] PATH_INFO: `/v3/users/4a0c2a52f0dd4cb0996d3f2733b74c4c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:36.063867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a0c2a52f0dd4cb0996d3f2733b74c4c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:36.065237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:36.067010 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:36.073337 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:36.120793 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d2ace3-f45b-406f-8186-723380c55f50 service placement] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:36.125683 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:36.126491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:36.126876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:36.127394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] PATH_INFO: `/v3/projects/6bd6d7b75f524b87aa47bab31516ba5c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:36.128888 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd6d7b75f524b87aa47bab31516ba5c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:36.130943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:36.292808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd6d7b75f524b87aa47bab31516ba5c', 'request_id': 'req-831c2b19-b6c8-47fc-a884-f05139a3d6a7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.294011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a0c2a52f0dd4cb0996d3f2733b74c4c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.300909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a0c2a52f0dd4cb0996d3f2733b74c4c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.392347 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd6d7b75f524b87aa47bab31516ba5c', 'request_id': 'req-831c2b19-b6c8-47fc-a884-f05139a3d6a7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.405616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd6d7b75f524b87aa47bab31516ba5c', 'request_id': 'req-831c2b19-b6c8-47fc-a884-f05139a3d6a7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.429037 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] The token cache is being invalidate because project 6bd6d7b75f524b87aa47bab31516ba5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:36.429464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.430287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.432167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1309/2586] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:36 2022] DELETE /identity/v3/projects/6bd6d7b75f524b87aa47bab31516ba5c => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:36.435737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:36.437355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:36.482619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a0c2a52f0dd4cb0996d3f2733b74c4c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.484544 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a0c2a52f0dd4cb0996d3f2733b74c4c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.506705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a0c2a52f0dd4cb0996d3f2733b74c4c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.512999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1278/2587] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:35 2022] DELETE /identity/v3/users/4a0c2a52f0dd4cb0996d3f2733b74c4c => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:36.514686 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831c2b19-b6c8-47fc-a884-f05139a3d6a7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:36.577919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:36.578662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:36.579017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:36.579302 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] PATH_INFO: `/v3/users/7c2f297fc6e24542b6bbb9f883d8b271` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:36.581252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2f297fc6e24542b6bbb9f883d8b271)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:36.583230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:36.754282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:36.756631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:36.771209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da40b1-f2e4-458c-8eea-ee22331688b3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:36.776710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:36.777576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:36.778052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:36.778487 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] PATH_INFO: `/v3/projects/818a0c6fa4a44406806a97d6de460c94` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:36.779648 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] RBAC: Authorizing `identity:delete_project(project_id=818a0c6fa4a44406806a97d6de460c94)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:36.781601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:36.804207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2f297fc6e24542b6bbb9f883d8b271'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.814281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2f297fc6e24542b6bbb9f883d8b271'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.815162 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2f297fc6e24542b6bbb9f883d8b271'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.815808 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2f297fc6e24542b6bbb9f883d8b271'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.838946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ef7827d8-b00b-4cc3-ade1-c4178350878b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2f297fc6e24542b6bbb9f883d8b271'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.944136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '818a0c6fa4a44406806a97d6de460c94', 'request_id': 'req-f2acf194-262a-41eb-bfd6-0413b90f7459'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.952161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '818a0c6fa4a44406806a97d6de460c94', 'request_id': 'req-f2acf194-262a-41eb-bfd6-0413b90f7459'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:36.958120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1310/2588] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:36 2022] DELETE /identity/v3/users/7c2f297fc6e24542b6bbb9f883d8b271 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:36.959185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '818a0c6fa4a44406806a97d6de460c94', 'request_id': 'req-f2acf194-262a-41eb-bfd6-0413b90f7459'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.022739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:37.072399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] The token cache is being invalidate because project 818a0c6fa4a44406806a97d6de460c94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:37.072721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.073386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f2acf194-262a-41eb-bfd6-0413b90f7459 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.076251 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1279/2589] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:36 2022] DELETE /identity/v3/projects/818a0c6fa4a44406806a97d6de460c94 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:37.082360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.082717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.082998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.107886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:37.109135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.174644 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1311/2590] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:37.182760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.183057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.183344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.204453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:37.206243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.218842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.220048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1280/2591] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:37.220562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.224861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.236851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f97f3-9bb9-4601-ba03-0858e84695ee None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.241854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.242633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.242949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b71fb9f-9d19-452a-b5f9-ba44f8d717f5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.243103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.243502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.244217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:37.244598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.246239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.250515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.251253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.251619 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.251908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] PATH_INFO: `/v3/projects/95f3fc21abff496e842436d4fb4bcccf` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.257275 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95f3fc21abff496e842436d4fb4bcccf)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.258187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.261759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1281/2592] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:37.265034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.277500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb587a4-99f7-46dd-a842-de3e2efcdd22 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.282926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.288405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.288405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.288405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.288405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.288405 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.372088 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95f3fc21abff496e842436d4fb4bcccf', 'request_id': 'req-577ef149-08e1-4d33-afff-1f4ea62c3379'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.388573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1282/2593] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/projects => generated 371 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:37.390681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.418020 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:37.451223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95f3fc21abff496e842436d4fb4bcccf', 'request_id': 'req-577ef149-08e1-4d33-afff-1f4ea62c3379'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.459541 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95f3fc21abff496e842436d4fb4bcccf', 'request_id': 'req-577ef149-08e1-4d33-afff-1f4ea62c3379'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.460019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5089ff-575d-45cb-994a-e95503953103 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.464877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.469463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.469463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.469463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.469463 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.470250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.476250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] The token cache is being invalidate because project 95f3fc21abff496e842436d4fb4bcccf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:37.478107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.478107 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-577ef149-08e1-4d33-afff-1f4ea62c3379 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:37.478733 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1312/2594] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:37 2022] DELETE /identity/v3/projects/95f3fc21abff496e842436d4fb4bcccf => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:37.483468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce0869-e0ea-497a-a784-1f059d645d6c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.483754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce0869-e0ea-497a-a784-1f059d645d6c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.484028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce0869-e0ea-497a-a784-1f059d645d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.505509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-4cce0869-e0ea-497a-a784-1f059d645d6c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:37.506750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4cce0869-e0ea-497a-a784-1f059d645d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.515080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1313/2595] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:37.523562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.523848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.524123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.534234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:37.544855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:37.546087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.554370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1314/2596] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:37.558367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.568452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22c87f-8338-4e4a-a3a8-f42cefd3f022 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.573551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.574259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.574672 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.574955 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1283/2597] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:37.575278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.575931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:37.576065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.576239 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.577590 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.577860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.591640 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f0f712-08d9-4984-976f-5bf4b824d6b9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.596146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1315/2598] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:18:37.597179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.597546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.598299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.598651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.599013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.599892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.601440 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.607566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6cb310-a3aa-49a1-87d2-16c389c3a62c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.613136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.617781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.617781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.617781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.617781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.618363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1284/2599] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:37.618476 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.618571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.632409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184005e1-6739-49ed-81bd-60f3ef05a89b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.637103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.637712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.638031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.638306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.638947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.640529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.656547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1285/2600] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:37.657782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.668635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d4299-516b-41a6-b44e-bcc067ad9c18 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.673363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b875527-2daa-44ba-bdd6-98c4808651f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.674684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b875527-2daa-44ba-bdd6-98c4808651f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.674999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b875527-2daa-44ba-bdd6-98c4808651f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.675283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b875527-2daa-44ba-bdd6-98c4808651f9 admin admin] PATH_INFO: `/v3/projects/6318fc1a44b149bca0e7ef4d1c482037/users/d54b00682ab5462ab2c97eeb5078be07/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.697041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b875527-2daa-44ba-bdd6-98c4808651f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6318fc1a44b149bca0e7ef4d1c482037, user_id=d54b00682ab5462ab2c97eeb5078be07, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.699763 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b875527-2daa-44ba-bdd6-98c4808651f9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.724967 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1316/2601] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/projects => generated 365 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:37.730354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.807778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8fb554-04be-496e-8f17-8ff3af69f421 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:37.811963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1286/2602] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:37 2022] PUT /identity/v3/projects/6318fc1a44b149bca0e7ef4d1c482037/users/d54b00682ab5462ab2c97eeb5078be07/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:37.831192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.831192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.831192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.868021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] MFA Rules not processed for user `d54b00682ab5462ab2c97eeb5078be07`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:37.869237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:37.895901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:37.896554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:37.896833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:37.897102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:37.897792 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:37.900184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:37.938737 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1287/2603] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:37.950210 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:37.951605 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:38.010965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06a6857-8ebe-48df-8e90-680e5b6b3174 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.011220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1317/2604] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:37 2022] POST /identity/v3/users => generated 283 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:38.015432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.018485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.019146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.019430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.019709 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.027309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155a8ba-5965-4cc8-b223-2215052ec00e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.028893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.030589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.032053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.032662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.033016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.033404 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.034250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.034386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:38.037492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.041475 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1288/2605] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:38.050093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1318/2606] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:38.054815 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.066422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069ea85-00e1-4a4e-9456-718760a0f18a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.069755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.073177 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.073820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.074115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.074393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.075041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.076651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.081852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9db9ac-9648-4d05-b845-84b6507651ad admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.086955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.087618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.088392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.088392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] PATH_INFO: `/v3/users/d54b00682ab5462ab2c97eeb5078be07` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.089294 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d54b00682ab5462ab2c97eeb5078be07)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.092929 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1289/2607] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:38.093110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.094657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.105827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711f286-7979-4a3b-83c7-827a816cf4e3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.113184 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc46ec7a-4d63-4a09-b8a0-87b3c4dda9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.113866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46ec7a-4d63-4a09-b8a0-87b3c4dda9a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.114232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46ec7a-4d63-4a09-b8a0-87b3c4dda9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.114657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc46ec7a-4d63-4a09-b8a0-87b3c4dda9a0 admin admin] PATH_INFO: `/v3/projects/82b86a959c714666aab52a47bd2dd545/users/7c44d92ffd774ace9b5f1df5707ede36/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.140919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46ec7a-4d63-4a09-b8a0-87b3c4dda9a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82b86a959c714666aab52a47bd2dd545, user_id=7c44d92ffd774ace9b5f1df5707ede36, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.145026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc46ec7a-4d63-4a09-b8a0-87b3c4dda9a0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.262964 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1290/2608] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:38 2022] PUT /identity/v3/projects/82b86a959c714666aab52a47bd2dd545/users/7c44d92ffd774ace9b5f1df5707ede36/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:38.275772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.276161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.276453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.302056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:38.309552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd54b00682ab5462ab2c97eeb5078be07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:38.309787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] MFA Rules not processed for user `7c44d92ffd774ace9b5f1df5707ede36`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:38.311152 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:38.319937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd54b00682ab5462ab2c97eeb5078be07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:38.320738 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd54b00682ab5462ab2c97eeb5078be07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:38.322120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd54b00682ab5462ab2c97eeb5078be07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:38.345502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-429760e1-5dc9-4866-a044-c4e172ebba6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd54b00682ab5462ab2c97eeb5078be07'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:38.386079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1291/2609] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:38.390702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.392387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:38.433167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1319/2610] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:38 2022] DELETE /identity/v3/users/d54b00682ab5462ab2c97eeb5078be07 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:38.437850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.441568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.441898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.463648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:38.465145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:38.476462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc5db6-3735-41ed-9ad9-22bacfb8783b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.545308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1320/2611] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:38.553577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.555266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.566174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.566174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.566503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.566611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.566714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:38.566808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.566808 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:38.621163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18458b7d-a00e-4648-accf-dc46ee968f10 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.625134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.625691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.625982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.626295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.636899 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.638421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.650600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1321/2612] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:38.780827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1292/2613] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:38 2022] POST /identity/v3/projects => generated 365 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:38.784331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.795030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3ca283-1ef6-48e3-871b-39240332fd5c service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.858513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.859283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:38.859574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:38.859858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:38.860505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:38.862635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:38.873260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:38.980756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1322/2614] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:38 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:38.982080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:38.993888 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3df7e5c-4237-4d8b-a8cc-aacc0c92fc8e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:38.998976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:38.999583 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.000018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.000304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.000948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.003114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.015359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1293/2615] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:39.019827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.030612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb7c0c-3e65-48d4-83b0-e4ecff0c82ba admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.036931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.037909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.038410 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.038748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] PATH_INFO: `/v3/projects/fe72b49539c946dc8d1d9b6f2bb9207a/users/4d2fd8e6e26b46a2a5b3de6c65963460/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.055721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe72b49539c946dc8d1d9b6f2bb9207a, user_id=4d2fd8e6e26b46a2a5b3de6c65963460, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.058479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.172828 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1323/2616] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:39 2022] PUT /identity/v3/projects/fe72b49539c946dc8d1d9b6f2bb9207a/users/4d2fd8e6e26b46a2a5b3de6c65963460/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:39.175855 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.187559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15b0a0b-fdf2-4d2e-b927-c6831896586f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.250596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.253420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.366106 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a281dc56-93f5-4b8f-b0bd-6c5a804f5edc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.371530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.373007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.373007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.373007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] PATH_INFO: `/v3/projects/6318fc1a44b149bca0e7ef4d1c482037` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.374574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6318fc1a44b149bca0e7ef4d1c482037)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.376734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.447789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.448555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.449305 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.449760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.450718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.452983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.470445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1294/2617] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:39 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:39.470445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.470445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.551325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa98ee3-15d0-4c65-8195-01c968b50586 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.555446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.556162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.556442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.556719 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.558607 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.572822 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.575007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.586984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6318fc1a44b149bca0e7ef4d1c482037', 'request_id': 'req-a9befd6e-44c7-4265-ac24-a03d99cd9de8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:39.601084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1295/2618] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:39.601914 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.613366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d124b1-b76a-4ed1-8593-083f310d2463 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.668416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6318fc1a44b149bca0e7ef4d1c482037', 'request_id': 'req-a9befd6e-44c7-4265-ac24-a03d99cd9de8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:39.691892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6318fc1a44b149bca0e7ef4d1c482037', 'request_id': 'req-a9befd6e-44c7-4265-ac24-a03d99cd9de8'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:39.692811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b43355a-5243-417d-be83-d3e646788f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.693502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43355a-5243-417d-be83-d3e646788f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.693860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43355a-5243-417d-be83-d3e646788f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.694201 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b43355a-5243-417d-be83-d3e646788f1b admin admin] PATH_INFO: `/v3/projects/fe72b49539c946dc8d1d9b6f2bb9207a/users/4d2fd8e6e26b46a2a5b3de6c65963460/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.696879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43355a-5243-417d-be83-d3e646788f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe72b49539c946dc8d1d9b6f2bb9207a, user_id=4d2fd8e6e26b46a2a5b3de6c65963460, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.701461 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b43355a-5243-417d-be83-d3e646788f1b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:39.717197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1296/2619] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:39 2022] PUT /identity/v3/projects/fe72b49539c946dc8d1d9b6f2bb9207a/users/4d2fd8e6e26b46a2a5b3de6c65963460/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:39.724369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] The token cache is being invalidate because project 6318fc1a44b149bca0e7ef4d1c482037 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:39.726804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:39.727941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9befd6e-44c7-4265-ac24-a03d99cd9de8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:39.730886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1324/2620] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:39 2022] DELETE /identity/v3/projects/6318fc1a44b149bca0e7ef4d1c482037 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:39.735540 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46842-429f-4df4-b440-a3fd7003eaaa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.735837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46842-429f-4df4-b440-a3fd7003eaaa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.736110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46842-429f-4df4-b440-a3fd7003eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.740561 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.741116 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.741495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.757675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:39.788848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] MFA Rules not processed for user `4d2fd8e6e26b46a2a5b3de6c65963460`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:39.790482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.807527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-0ce46842-429f-4df4-b440-a3fd7003eaaa None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:39.808765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0ce46842-429f-4df4-b440-a3fd7003eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.897042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1297/2621] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:39.901474 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:39.906299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.907783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.916451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1325/2622] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:39.927368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.927898 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.928422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.951958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:39.953543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:39.967053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1326/2623] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:39.972940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:39.984314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b732ee-0fc7-40ea-b4f4-ed7a538131b0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.991448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.994616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.994766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4483f-d684-4537-b3cb-0b0922727023 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:39.994931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:39.995742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:39.997033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:39.997654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:39.998857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:39.999627 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:39.999931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.000350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.002994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.014554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.019167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.020519 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1327/2624] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:40.026021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.048349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb13916-b527-4bf8-8efe-c35547b1235a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.051812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1298/2625] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:40.065590 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.067005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.067665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.068137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.069848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.075647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.227876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1328/2626] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:40 2022] POST /identity/v3/projects => generated 371 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:40.228793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.256585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431e48df-f573-4802-91ff-a9c3163a8098 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.334602 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.335885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.336264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.336554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.337650 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.339880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.474558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1299/2627] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:40 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:40.478884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.492571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911230d-d36d-4a2f-8438-bcb68fa6520e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.497609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.498490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.499093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.499614 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.500462 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.502812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.518851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.520180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1329/2628] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:40.530141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f40ce1-4ef4-434b-8275-662ee535d7a9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.554437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19448d8-0ba9-4e7c-a179-60e29d096e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.556040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19448d8-0ba9-4e7c-a179-60e29d096e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.556339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19448d8-0ba9-4e7c-a179-60e29d096e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.556625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19448d8-0ba9-4e7c-a179-60e29d096e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.557282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19448d8-0ba9-4e7c-a179-60e29d096e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.572771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19448d8-0ba9-4e7c-a179-60e29d096e07 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.579094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1300/2629] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:40.583992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.594422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b9d56-1794-44f4-90f4-92d77dd298df admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:40.599942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:40.600612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.600889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.601180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] PATH_INFO: `/v3/projects/e328fc06dc19416a889a1f08aef03943/users/f3ca11c79c1b447494718d70a8a8cfa6/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.618815 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e328fc06dc19416a889a1f08aef03943, user_id=f3ca11c79c1b447494718d70a8a8cfa6, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:40.621488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:40.754510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1330/2630] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:40 2022] PUT /identity/v3/projects/e328fc06dc19416a889a1f08aef03943/users/f3ca11c79c1b447494718d70a8a8cfa6/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:40.765200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:40.765497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:40.765810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:40.793386 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:40.796966 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] MFA Rules not processed for user `f3ca11c79c1b447494718d70a8a8cfa6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:40.801081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:40.907347 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1301/2631] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:40.919195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:40.921642 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:41.038934 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de74a0b9-14f1-4adb-9039-82c27d760efd admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:41.045116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec99fe-eccf-4917-ad3c-349b678de5cd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:41.045826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec99fe-eccf-4917-ad3c-349b678de5cd service glance] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:41.046066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec99fe-eccf-4917-ad3c-349b678de5cd service glance] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:41.046346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec99fe-eccf-4917-ad3c-349b678de5cd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:41.059226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec99fe-eccf-4917-ad3c-349b678de5cd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:41.060705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec99fe-eccf-4917-ad3c-349b678de5cd service glance] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:41.072285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1331/2632] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:41.343478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:41.345223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:41.536001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c91458-dd0e-4196-bff3-32e1c87824ce None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:41.541897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:41.546306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:41.546306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:41.546306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] PATH_INFO: `/v3/users/7c44d92ffd774ace9b5f1df5707ede36` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:41.546306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c44d92ffd774ace9b5f1df5707ede36)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:41.546830 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:41.841845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:41.850397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c44d92ffd774ace9b5f1df5707ede36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:41.874228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c44d92ffd774ace9b5f1df5707ede36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:42.005231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c44d92ffd774ace9b5f1df5707ede36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:42.006019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c44d92ffd774ace9b5f1df5707ede36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:42.021083 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c44d92ffd774ace9b5f1df5707ede36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:42.028285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1302/2633] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:41 2022] DELETE /identity/v3/users/7c44d92ffd774ace9b5f1df5707ede36 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:42.032985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:42.034242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:42.045901 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:42.071756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:42.074279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:42.164786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1332/2634] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:42.179092 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:42.180893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:42.322885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8797b484-bfed-4921-9a3e-1030680510ef admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:42.327019 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:42.327767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:42.328095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:42.328334 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:42.342272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:42.342390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:42.360863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1303/2635] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:42.957080 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:42.958879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:42.972632 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58467e37-997d-4eb6-954e-b471151c04b3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:42.979533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:42.981394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:42.981394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:42.981394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] PATH_INFO: `/v3/projects/82b86a959c714666aab52a47bd2dd545` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:42.982066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82b86a959c714666aab52a47bd2dd545)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:42.984041 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:43.006698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:43.129265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82b86a959c714666aab52a47bd2dd545', 'request_id': 'req-61f5d13d-71b0-4980-8d0b-334142e18e04'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.233958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82b86a959c714666aab52a47bd2dd545', 'request_id': 'req-61f5d13d-71b0-4980-8d0b-334142e18e04'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.244467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82b86a959c714666aab52a47bd2dd545', 'request_id': 'req-61f5d13d-71b0-4980-8d0b-334142e18e04'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.262963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] The token cache is being invalidate because project 82b86a959c714666aab52a47bd2dd545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:43.263873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.264836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.267163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1333/2636] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:42 2022] DELETE /identity/v3/projects/82b86a959c714666aab52a47bd2dd545 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:43.271940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:43.282223 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f932da-6588-42e6-ae2d-8c899af4ca78 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:43.383312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:43.383956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:43.384247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:43.384527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] PATH_INFO: `/v3/users/4d2fd8e6e26b46a2a5b3de6c65963460` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:43.385858 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2fd8e6e26b46a2a5b3de6c65963460)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:43.388011 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:43.708015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2fd8e6e26b46a2a5b3de6c65963460'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.711002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:43.719577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2fd8e6e26b46a2a5b3de6c65963460'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.816517 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2fd8e6e26b46a2a5b3de6c65963460'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.817308 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2fd8e6e26b46a2a5b3de6c65963460'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.832248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2fd8e6e26b46a2a5b3de6c65963460'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:43.839072 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1304/2637] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:43 2022] DELETE /identity/v3/users/4d2fd8e6e26b46a2a5b3de6c65963460 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:43.872211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:43.873786 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:43.888664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5d13d-71b0-4980-8d0b-334142e18e04 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:43.972635 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:43.973290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:43.976216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:43.976216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] PATH_INFO: `/v3/users/2ffe5c7cb44f476e913d64003105002c` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:43.976216 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffe5c7cb44f476e913d64003105002c)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:43.976940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.152493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffe5c7cb44f476e913d64003105002c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.160712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffe5c7cb44f476e913d64003105002c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.165398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffe5c7cb44f476e913d64003105002c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.166168 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffe5c7cb44f476e913d64003105002c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.179533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffe5c7cb44f476e913d64003105002c'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.234081 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1334/2638] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:43 2022] DELETE /identity/v3/users/2ffe5c7cb44f476e913d64003105002c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:44.301686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.303369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.321669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aea131-3910-41db-b4f2-098442ea8545 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.423167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.424035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.424662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.426052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] PATH_INFO: `/v3/projects/fe72b49539c946dc8d1d9b6f2bb9207a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.428035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe72b49539c946dc8d1d9b6f2bb9207a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.431193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.545240 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.546850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.564486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a003d597-2315-4eb6-8894-30bd1c877df9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.572652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.573548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.574062 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.574413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] PATH_INFO: `/v3/projects/742c089b0dea4b1f89efd31f48f9ffc5` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.575641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=742c089b0dea4b1f89efd31f48f9ffc5)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.577561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.593076 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe72b49539c946dc8d1d9b6f2bb9207a', 'request_id': 'req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.596388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:44.600257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe72b49539c946dc8d1d9b6f2bb9207a', 'request_id': 'req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.607823 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe72b49539c946dc8d1d9b6f2bb9207a', 'request_id': 'req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.705890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742c089b0dea4b1f89efd31f48f9ffc5', 'request_id': 'req-3aa1ee29-4d27-499f-b798-f84e82c51e47'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.713257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] The token cache is being invalidate because project fe72b49539c946dc8d1d9b6f2bb9207a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:44.715004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.715004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad0ea9cc-e8c7-4e55-a891-d44981e3ee5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.715776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1305/2639] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:44 2022] DELETE /identity/v3/projects/fe72b49539c946dc8d1d9b6f2bb9207a => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:44.722868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3e422-46fd-400f-9ae4-736b23501ad0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.723209 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3e422-46fd-400f-9ae4-736b23501ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.723485 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3e422-46fd-400f-9ae4-736b23501ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.773460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-10f3e422-46fd-400f-9ae4-736b23501ad0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.774682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-10f3e422-46fd-400f-9ae4-736b23501ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.808025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:44.822503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742c089b0dea4b1f89efd31f48f9ffc5', 'request_id': 'req-3aa1ee29-4d27-499f-b798-f84e82c51e47'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.829643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742c089b0dea4b1f89efd31f48f9ffc5', 'request_id': 'req-3aa1ee29-4d27-499f-b798-f84e82c51e47'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.841753 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] The token cache is being invalidate because project 742c089b0dea4b1f89efd31f48f9ffc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:44.842064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.849199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1306/2640] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:44.849407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-3aa1ee29-4d27-499f-b798-f84e82c51e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:44.849407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1335/2641] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:44 2022] DELETE /identity/v3/projects/742c089b0dea4b1f89efd31f48f9ffc5 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:44.849829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a6719-64c6-4175-86b5-2392db44a04b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.850065 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a6719-64c6-4175-86b5-2392db44a04b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.850343 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a6719-64c6-4175-86b5-2392db44a04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.851954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.852254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.852536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.871947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5e5a6719-64c6-4175-86b5-2392db44a04b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.873175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e5a6719-64c6-4175-86b5-2392db44a04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.877542 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.879636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.881307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1307/2642] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:44.888339 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1336/2643] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:44.889285 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.889851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.895919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.898772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.908533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc25b43-1990-4484-977b-2f1955917d36 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.912999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:44.914323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:44.915196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.915819 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.916105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.916384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.917069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:44.917372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.919000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.923087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1308/2644] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:44.928060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.933317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1337/2645] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:18:44.937191 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f275722e-eb0b-45c3-9dc8-ed72e5ab03b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.937388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.942158 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.943018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.944597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.944597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.944831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:44.945128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.946761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.947497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93c8cf-1c46-4aed-bd56-f22cd3c4b17b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.952544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.953042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.953489 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.953706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.954479 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.956531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:44.960157 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1309/2646] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:18:44.964726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:44.977656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0b3a1-bb68-4437-9360-6e71869a3ed4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:44.983340 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:44.990389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:44.990389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:44.990389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:44.990389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:44.990389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.091544 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1338/2647] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:45.095151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.111322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1310/2648] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:44 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:45.113037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.179584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2aaa31-df48-41f2-9ceb-6e5b14b2350b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.193330 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75038a0c-0129-4f03-9970-c1c4c56fde1b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.198042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.198646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.198925 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.199236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.199861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.201805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.241424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.242158 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.242458 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.242724 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.243378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.245367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.277980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1311/2649] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:45.283008 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.294179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367f2c3-2313-4d2c-b917-516e93b1d71b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.298839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.299436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.299751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.300018 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.300682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.302315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.312491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1312/2650] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:45.316593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.326558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132bf4e3-9997-4a83-9921-55386c80a92c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.331720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.332323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.332598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.332865 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.333495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.335142 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.343696 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:45.345900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1313/2651] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:45.350643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.351361 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1339/2652] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/users => generated 287 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:45.355248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.365066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb4a9b-66de-4f7b-a611-3dfa0971e3eb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.365700 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0794524-ea67-4930-a0a9-7ed3d139c052 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.370373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.370534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba40a2f-ab10-4a20-87d3-f4284c1a3e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.370977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.371139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba40a2f-ab10-4a20-87d3-f4284c1a3e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.371244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.371395 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba40a2f-ab10-4a20-87d3-f4284c1a3e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.371517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.371670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba40a2f-ab10-4a20-87d3-f4284c1a3e09 admin admin] PATH_INFO: `/v3/projects/884db33c970a481f9badd9867fa482f1/users/77e1f5c7c7cc40eea03f4cd172548761/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.372156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.373817 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.384301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1340/2653] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:45.387631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba40a2f-ab10-4a20-87d3-f4284c1a3e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884db33c970a481f9badd9867fa482f1, user_id=77e1f5c7c7cc40eea03f4cd172548761, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.388408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.390371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba40a2f-ab10-4a20-87d3-f4284c1a3e09 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.399533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647657c9-04e6-479a-9b78-75d296f1ab71 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.405865 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.406511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.406796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.407071 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.407717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.409950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.421091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1341/2654] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:45.425429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.435431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55541a-e1d7-4c84-b159-ef35bd69cbb6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.441262 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2befed8-3b8a-40e4-b0d1-51fccae7122b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.442019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2befed8-3b8a-40e4-b0d1-51fccae7122b admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.442307 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2befed8-3b8a-40e4-b0d1-51fccae7122b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.442583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2befed8-3b8a-40e4-b0d1-51fccae7122b admin admin] PATH_INFO: `/v3/projects/0cebcfd778a74044beb300924992f5fa/users/01b404bef2f74ce7bfb767be1e0cbaed/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.458123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2befed8-3b8a-40e4-b0d1-51fccae7122b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cebcfd778a74044beb300924992f5fa, user_id=01b404bef2f74ce7bfb767be1e0cbaed, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.460800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2befed8-3b8a-40e4-b0d1-51fccae7122b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.474050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1314/2655] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:45 2022] PUT /identity/v3/projects/884db33c970a481f9badd9867fa482f1/users/77e1f5c7c7cc40eea03f4cd172548761/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:45.485952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.486243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.486522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.515743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] MFA Rules not processed for user `77e1f5c7c7cc40eea03f4cd172548761`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:45.517005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:45.534626 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:45.536591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1342/2656] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:45 2022] PUT /identity/v3/projects/0cebcfd778a74044beb300924992f5fa/users/01b404bef2f74ce7bfb767be1e0cbaed/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:45.549528 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.550226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.550563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.586600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] MFA Rules not processed for user `01b404bef2f74ce7bfb767be1e0cbaed`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:45.589077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:45.613064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1315/2657] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:45.622543 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.624060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:45.668486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1343/2658] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:45.673109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.734411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d0093-9046-4d2a-b19c-1b60a5039e76 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.740505 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c570bfc-4af1-4c84-920c-c897055bc68e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.745279 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.750252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.750252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.750252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.750252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.750252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.800921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.801617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.801947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.802208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.858266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1344/2659] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/projects => generated 375 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:45.861166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:45.863663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.865371 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:45.874895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1316/2660] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:45.903171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:45.931288 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:45.940447 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad342a-fc63-423e-9009-4951f8622c6f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:45.947762 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:45.948423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:45.948707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:45.948984 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:45.949654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:45.951832 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.052699 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1345/2661] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:45 2022] POST /identity/v3/users => generated 294 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:46.056751 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.069217 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf06d8c-7275-4acc-9eb5-a5ebd9d8a27f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.074141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:46.074752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.075043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.075320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.075959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:46.077546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.088481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1317/2662] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:46.092775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.103629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02282efc-19f1-4947-9240-918a69267f7f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.108688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:46.109268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.109553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.109906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] PATH_INFO: `/v3/projects/9946f2912838467eb1f244163586a148/users/d940baa4460041e1bf7035edfe1dbad5/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.127246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9946f2912838467eb1f244163586a148, user_id=d940baa4460041e1bf7035edfe1dbad5, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:46.130778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.208017 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1346/2663] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:46 2022] PUT /identity/v3/projects/9946f2912838467eb1f244163586a148/users/d940baa4460041e1bf7035edfe1dbad5/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:46.212435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.222903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574c1e10-2f6c-405b-881f-e9e39ad1eab3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.295287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0066eeb1-8e07-4d1e-a6ac-eaf2ca3c0669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:46.296038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066eeb1-8e07-4d1e-a6ac-eaf2ca3c0669 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.296322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066eeb1-8e07-4d1e-a6ac-eaf2ca3c0669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.296598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0066eeb1-8e07-4d1e-a6ac-eaf2ca3c0669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.297242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066eeb1-8e07-4d1e-a6ac-eaf2ca3c0669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:46.298872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0066eeb1-8e07-4d1e-a6ac-eaf2ca3c0669 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.312827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1318/2664] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:46.315042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.326787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec1308-3884-4a74-b0ea-46402ccae98d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.332348 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:46.333151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.333432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.333719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] PATH_INFO: `/v3/projects/9946f2912838467eb1f244163586a148/users/d940baa4460041e1bf7035edfe1dbad5/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.335755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9946f2912838467eb1f244163586a148, user_id=d940baa4460041e1bf7035edfe1dbad5, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:46.338558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.349742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1347/2665] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:46 2022] PUT /identity/v3/projects/9946f2912838467eb1f244163586a148/users/d940baa4460041e1bf7035edfe1dbad5/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:46.369396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.369603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.369913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.402617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] MFA Rules not processed for user `d940baa4460041e1bf7035edfe1dbad5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:46.403857 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:46.483917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1319/2666] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:46 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:46.495953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:46.497715 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:46.562188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3405a-7f20-488d-b55a-b03b671f7f74 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:46.567061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec730722-07b7-4f7d-86fb-5787cb4e17bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:46.571561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec730722-07b7-4f7d-86fb-5787cb4e17bf service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:46.571561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec730722-07b7-4f7d-86fb-5787cb4e17bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:46.571561 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec730722-07b7-4f7d-86fb-5787cb4e17bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:46.635943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec730722-07b7-4f7d-86fb-5787cb4e17bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:46.637401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec730722-07b7-4f7d-86fb-5787cb4e17bf service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:46.647840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1348/2667] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:52.827037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:52.875936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:52.901356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef2722-ee37-478b-b6c7-44a0b760a1df None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:52.906275 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:52.907323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:52.907323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:52.907513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] PATH_INFO: `/v3/users/01b404bef2f74ce7bfb767be1e0cbaed` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:52.908934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01b404bef2f74ce7bfb767be1e0cbaed)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:52.911862 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:53.122445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01b404bef2f74ce7bfb767be1e0cbaed'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.130160 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01b404bef2f74ce7bfb767be1e0cbaed'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.221869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01b404bef2f74ce7bfb767be1e0cbaed'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.222500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01b404bef2f74ce7bfb767be1e0cbaed'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.235299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01b404bef2f74ce7bfb767be1e0cbaed'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:53.242297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1320/2668] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:52 2022] DELETE /identity/v3/users/01b404bef2f74ce7bfb767be1e0cbaed => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:53.247198 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:53.247745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:53.247853 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:53.268718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:53.269962 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:53.307735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:53.471905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1349/2669] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:53.481677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:53.483249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:53.543306 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fc6264-ce7e-4962-acad-d16408a02787 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:53.546891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:53.547495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:53.547777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:53.548051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:53.558364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:53.559836 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:53.571119 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1321/2670] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:54.043842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.045340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:54.058938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc0514-f64b-453c-9867-57c4f3b221c0 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.065766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.065766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.065766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.066136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] PATH_INFO: `/v3/projects/0cebcfd778a74044beb300924992f5fa` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.067365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cebcfd778a74044beb300924992f5fa)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.069449 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.182283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cebcfd778a74044beb300924992f5fa', 'request_id': 'req-7f729619-d573-41d5-b39e-f57c821a6954'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.284533 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cebcfd778a74044beb300924992f5fa', 'request_id': 'req-7f729619-d573-41d5-b39e-f57c821a6954'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.293246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cebcfd778a74044beb300924992f5fa', 'request_id': 'req-7f729619-d573-41d5-b39e-f57c821a6954'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.304396 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] The token cache is being invalidate because project 0cebcfd778a74044beb300924992f5fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:54.304698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.305349 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.307562 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1350/2671] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:54 2022] DELETE /identity/v3/projects/0cebcfd778a74044beb300924992f5fa => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:54.311782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:54.322369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7162b3-bffc-460e-a077-772cae785042 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:54.388490 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:54.389253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:54.389542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:54.389845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] PATH_INFO: `/v3/users/d940baa4460041e1bf7035edfe1dbad5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:54.391027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d940baa4460041e1bf7035edfe1dbad5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:54.393027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:54.394800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:54.592527 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd940baa4460041e1bf7035edfe1dbad5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.600938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd940baa4460041e1bf7035edfe1dbad5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.717392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd940baa4460041e1bf7035edfe1dbad5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.718087 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd940baa4460041e1bf7035edfe1dbad5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.731163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd940baa4460041e1bf7035edfe1dbad5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:54.737027 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1322/2672] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:54 2022] DELETE /identity/v3/users/d940baa4460041e1bf7035edfe1dbad5 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:55.231454 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.232646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.246760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f729619-d573-41d5-b39e-f57c821a6954 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.308492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.309203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.309455 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.309748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] PATH_INFO: `/v3/users/77e1f5c7c7cc40eea03f4cd172548761` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.311310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] RBAC: Authorizing `identity:delete_user(user_id=77e1f5c7c7cc40eea03f4cd172548761)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.313840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.503968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.504614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.521992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4423f27-7d7d-4a0d-b43a-bfb0f230ae70 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.527912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.529055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.529399 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.529875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] PATH_INFO: `/v3/projects/9946f2912838467eb1f244163586a148` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.531325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9946f2912838467eb1f244163586a148)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.533465 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.572587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77e1f5c7c7cc40eea03f4cd172548761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.579829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77e1f5c7c7cc40eea03f4cd172548761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.580510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77e1f5c7c7cc40eea03f4cd172548761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.581130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77e1f5c7c7cc40eea03f4cd172548761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.586555 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:55.601253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1edb3a62-ad36-47aa-a7c4-8f3a9f39078f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77e1f5c7c7cc40eea03f4cd172548761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.698639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1351/2673] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:55 2022] DELETE /identity/v3/users/77e1f5c7c7cc40eea03f4cd172548761 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:55.699757 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9946f2912838467eb1f244163586a148', 'request_id': 'req-f61c5fe5-e77b-4af3-857b-140bb58b3817'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.706437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.708219 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9946f2912838467eb1f244163586a148', 'request_id': 'req-f61c5fe5-e77b-4af3-857b-140bb58b3817'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.708768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.709271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.716215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9946f2912838467eb1f244163586a148', 'request_id': 'req-f61c5fe5-e77b-4af3-857b-140bb58b3817'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.751256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:55.752478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.821383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1352/2674] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:55.834351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.835765 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.847303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] The token cache is being invalidate because project 9946f2912838467eb1f244163586a148 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:55.847641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.848805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f61c5fe5-e77b-4af3-857b-140bb58b3817 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:55.850365 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1323/2675] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:55 2022] DELETE /identity/v3/projects/9946f2912838467eb1f244163586a148 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:55.854991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee46cd-f225-46f7-9098-ab71535a44cd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.855408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee46cd-f225-46f7-9098-ab71535a44cd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.855760 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee46cd-f225-46f7-9098-ab71535a44cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.876701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d6ee46cd-f225-46f7-9098-ab71535a44cd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:55.878199 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d6ee46cd-f225-46f7-9098-ab71535a44cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.887125 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1324/2676] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:55.895680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.895972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.896257 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.906429 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf887607-15c2-4c93-ad96-16d3d0827b74 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.910232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.912647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.912929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.913201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.915101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.918088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:55.919293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:55.927737 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.927923 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1325/2677] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:55.929270 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.932501 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.939714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1353/2678] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:55.945150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3745b15d-5bd9-4800-865f-40b8510e8036 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.950175 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.950789 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.951071 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.951344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.951998 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:55.952300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.953928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:55.966779 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1326/2679] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:18:55.971174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:55.981543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ccb5-13ce-4ff3-8967-62898be71c4b service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:55.986571 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:55.987170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:55.987499 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:55.987969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:55.988445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:55.990432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.136570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1354/2680] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:55 2022] POST /identity/v3/projects => generated 385 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:56.140654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.151265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f350f5a-c4a8-46a5-9b16-1ea46d7b275b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.191252 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:56.227698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.233993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.233993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.233993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.233993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.233993 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.371549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1327/2681] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:56.375638 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.387008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b55130-3dc7-41f8-9220-3f6b5093a06d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.391617 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.392229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.392508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.392781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.393421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.395148 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.406657 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1355/2682] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:56.417473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.417473 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.418154 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.427671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a629c-7fae-45ab-aec9-ade24521ea18 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.429651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f7141e-8493-491c-a500-b2b0da417c69 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.431475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.432285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.432445 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.432591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.433237 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.435012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.438049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.439196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.439807 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.440216 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] PATH_INFO: `/v3/projects/884db33c970a481f9badd9867fa482f1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.443908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] RBAC: Authorizing `identity:delete_project(project_id=884db33c970a481f9badd9867fa482f1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.451798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1356/2683] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:56.452668 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.453319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.463712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf5f3d-e58f-4ea1-84c0-c8e1905a0c72 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.471186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.471847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.473838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.473838 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] PATH_INFO: `/v3/projects/5b93bc70d91e4761bc3cad29d8061825/users/684a8f7a5cda4a1688cc75fc0ef0d788/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.490661 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b93bc70d91e4761bc3cad29d8061825, user_id=684a8f7a5cda4a1688cc75fc0ef0d788, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.493337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.589855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1357/2684] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:56 2022] PUT /identity/v3/projects/5b93bc70d91e4761bc3cad29d8061825/users/684a8f7a5cda4a1688cc75fc0ef0d788/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:56.590801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.592407 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.604104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884db33c970a481f9badd9867fa482f1', 'request_id': 'req-65f578a0-5aae-4c0f-b2ef-b07750abd19f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:56.610921 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884db33c970a481f9badd9867fa482f1', 'request_id': 'req-65f578a0-5aae-4c0f-b2ef-b07750abd19f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:56.612467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:56.618498 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884db33c970a481f9badd9867fa482f1', 'request_id': 'req-65f578a0-5aae-4c0f-b2ef-b07750abd19f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:56.684675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] The token cache is being invalidate because project 884db33c970a481f9badd9867fa482f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:56.684920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:56.685666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65f578a0-5aae-4c0f-b2ef-b07750abd19f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:56.687213 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1328/2685] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:56 2022] DELETE /identity/v3/projects/884db33c970a481f9badd9867fa482f1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:56.687894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ad84d-031a-487a-9a69-ebd16718f735 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.691912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.695050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.695050 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.720842 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] MFA Rules not processed for user `684a8f7a5cda4a1688cc75fc0ef0d788`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:56.722139 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.755260 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.755876 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.756157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.756430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.758346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.771394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.772991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4fd8aa-b064-4f82-a998-c152a4881186 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.783636 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1358/2686] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:56.784927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3812847-6565-41f7-a712-6ef85602c53c None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.785217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3812847-6565-41f7-a712-6ef85602c53c None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.786012 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3812847-6565-41f7-a712-6ef85602c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.803238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1329/2687] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:56.804749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.806711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.807945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-d3812847-6565-41f7-a712-6ef85602c53c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:56.809113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d3812847-6565-41f7-a712-6ef85602c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.852235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:56.872961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1359/2688] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:56.878004 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b6043-d5c4-48e6-ab80-28560e7394f0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.880881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.881176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.881453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.884009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.884692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.885141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.885886 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.886652 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.888633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.903500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:56.904705 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:56.920187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1360/2689] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:18:56.921771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.931892 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef7323-aac0-41a1-a714-f00030842ceb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.936956 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.937523 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.937824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.938101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.938789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:18:56.939094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.940745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.959278 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1361/2690] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:18:56.960595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.973537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62482ea-64b7-4b3f-b870-8e977a05494f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:56.978104 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1330/2691] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/projects => generated 383 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:56.979620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:56.980192 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:56.980772 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:56.981010 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:56.981351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:56.981958 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:56.983873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:56.989682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:57.057906 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63202b2-f8b7-4abb-b496-44fcf65902d0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.063073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.063798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.064084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.064357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.064983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.067990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.076631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1362/2692] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:57.081340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.173278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1331/2693] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:56 2022] POST /identity/v3/users => generated 292 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:57.173278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.175943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b7750-9a1c-4738-a2fd-76bf4ecc1899 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.187176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad28f1d-158b-468f-86bb-e1b6a6a9faba admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.189560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.190229 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.190515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.190799 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.192330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.194556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.194788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.195549 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.196000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.196445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.197350 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.199260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.214381 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1332/2694] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:57.215613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.228207 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f231d-72c5-4299-a055-9d76d69af7bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.234021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.235016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.235304 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.235576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.236228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.237926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.252962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1333/2695] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:57.262194 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.273015 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb56a8f-ea6f-4d38-b217-34dc2103d1f0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.277876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.278535 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.278933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.279311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] PATH_INFO: `/v3/projects/a5b647fedc864215a8bcf90a1738ef62/users/4bffad21692a4cd1bf17ab0564fa520d/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.284953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1363/2696] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:57 2022] POST /identity/v3/users => generated 288 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:57.286056 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.288091 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:57.296248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5b647fedc864215a8bcf90a1738ef62, user_id=4bffad21692a4cd1bf17ab0564fa520d, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.299043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.346969 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852d5c0-e81a-4e88-99b2-010efed2596e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.350940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.351550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.351835 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.352112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.353939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:57.395147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1334/2697] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:57 2022] PUT /identity/v3/projects/a5b647fedc864215a8bcf90a1738ef62/users/4bffad21692a4cd1bf17ab0564fa520d/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:57.404657 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.423968 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.425420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dacc8c-f066-4095-9891-f440106bdbc2 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.436851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1364/2698] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:57.450840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.450840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.450840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.467940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] MFA Rules not processed for user `4bffad21692a4cd1bf17ab0564fa520d`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:57.469171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:57.486215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:57.491038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc1cd1-2ffa-4581-99b7-7321725247b7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.498729 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.499382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.499661 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.499935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.500579 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.502234 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.519052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1335/2699] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:57.520471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.526875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:57.530596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ecb02-7d86-47a7-9515-f81e20b99931 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.536156 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.536769 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.537295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.537577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.538236 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.539812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.550128 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1365/2700] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:57.553783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1336/2701] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:57.555430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.555635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.565587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309f238-e5a3-4c3c-a87b-d49f5b138c52 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.566175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f63ef-5596-41b4-8cf4-2aa05549660b None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.570497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982d3e75-fe0a-4ca2-aaff-d7fb3b4c157c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.571147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3e75-fe0a-4ca2-aaff-d7fb3b4c157c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.571301 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.571428 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3e75-fe0a-4ca2-aaff-d7fb3b4c157c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.571688 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982d3e75-fe0a-4ca2-aaff-d7fb3b4c157c admin admin] PATH_INFO: `/v3/projects/2ecb2b23305249c7aa6c328ef0dafc99/users/2fbbbb374d284e9b9ad2e490c7aaa1ea/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.572093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.572443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.572783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.573521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.575553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.589245 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d3e75-fe0a-4ca2-aaff-d7fb3b4c157c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ecb2b23305249c7aa6c328ef0dafc99, user_id=2fbbbb374d284e9b9ad2e490c7aaa1ea, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.592882 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982d3e75-fe0a-4ca2-aaff-d7fb3b4c157c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.663579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1366/2702] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:18:57 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:18:57.667991 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.678743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1337/2703] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:57 2022] PUT /identity/v3/projects/2ecb2b23305249c7aa6c328ef0dafc99/users/2fbbbb374d284e9b9ad2e490c7aaa1ea/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:57.689095 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.689496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.689927 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.728204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] MFA Rules not processed for user `2fbbbb374d284e9b9ad2e490c7aaa1ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:57.729497 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:57.752746 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4248978-0ed1-464c-b0dc-dbf0d9362120 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.812588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1338/2704] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:57.823050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.823674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.823955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.824162 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.824273 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.824860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.826064 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:57.826874 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.907852 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc9d389-7997-430f-8310-e5836cf3e0c6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.915202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.915832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.916118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.916403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.919976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1367/2705] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:18:57 2022] POST /identity/v3/users => generated 298 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:18:57.924977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.928225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.930056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.938152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71f54c-3500-46ce-a54f-ae3f9f5617c9 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.939849 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1339/2706] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:57.953194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.953927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.955855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.955855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:57.955855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:57.956768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:57.970102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1368/2707] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:18:57.979037 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:57.989278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d492cf-b8c6-4761-867d-0a9a60c7c82e service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:57.992587 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:57.993270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:57.993552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:57.993866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] PATH_INFO: `/v3/projects/d8a13e8edcfe4f119e4033bc117a0924/users/f50ac7590c1a4ec5b23ff3673fed31e7/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.010910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a13e8edcfe4f119e4033bc117a0924, user_id=f50ac7590c1a4ec5b23ff3673fed31e7, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.013577 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.123503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1340/2708] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:57 2022] PUT /identity/v3/projects/d8a13e8edcfe4f119e4033bc117a0924/users/f50ac7590c1a4ec5b23ff3673fed31e7/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:58.127920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:58.139491 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44de6a41-8231-437b-bcb1-31d162d52c5b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:58.161588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:58.168109 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:58.197860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:58.221976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:58.222825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.228395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.228395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.228395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.228395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.237994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1369/2709] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:18:58 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:18:58.245182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:58.256895 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89579a71-087e-4e13-9bfe-78afcddc7ad4 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:58.262868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d104ec-bc16-404e-acb4-cf591c6f589c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:58.263521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d104ec-bc16-404e-acb4-cf591c6f589c admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.263872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d104ec-bc16-404e-acb4-cf591c6f589c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.264171 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d104ec-bc16-404e-acb4-cf591c6f589c admin admin] PATH_INFO: `/v3/projects/d8a13e8edcfe4f119e4033bc117a0924/users/f50ac7590c1a4ec5b23ff3673fed31e7/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.266185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d104ec-bc16-404e-acb4-cf591c6f589c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a13e8edcfe4f119e4033bc117a0924, user_id=f50ac7590c1a4ec5b23ff3673fed31e7, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.268825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d104ec-bc16-404e-acb4-cf591c6f589c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.277675 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfda4d-ef87-4e99-b464-b37f187ad6e0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:58.293509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1370/2710] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:18:58 2022] PUT /identity/v3/projects/d8a13e8edcfe4f119e4033bc117a0924/users/f50ac7590c1a4ec5b23ff3673fed31e7/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:58.295221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.295508 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.295785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.334603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] MFA Rules not processed for user `f50ac7590c1a4ec5b23ff3673fed31e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:58.335531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:58.350659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:58.351310 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.351591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.351867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] PATH_INFO: `/v3/users/2fbbbb374d284e9b9ad2e490c7aaa1ea` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.353040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fbbbb374d284e9b9ad2e490c7aaa1ea)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.355023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.414208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1371/2711] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:58.426654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:58.428663 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:58.496488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347fb867-1a6f-4a01-b885-25adc25610f5 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:58.500218 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee319596-4587-42da-b42f-78d114c24a80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:58.500810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee319596-4587-42da-b42f-78d114c24a80 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.501090 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee319596-4587-42da-b42f-78d114c24a80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.501365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee319596-4587-42da-b42f-78d114c24a80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.544325 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:58.556999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fbbbb374d284e9b9ad2e490c7aaa1ea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.561645 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee319596-4587-42da-b42f-78d114c24a80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.563130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee319596-4587-42da-b42f-78d114c24a80 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.564100 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fbbbb374d284e9b9ad2e490c7aaa1ea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.574754 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1372/2712] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:18:58.650256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fbbbb374d284e9b9ad2e490c7aaa1ea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.650874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fbbbb374d284e9b9ad2e490c7aaa1ea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.674649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fbbbb374d284e9b9ad2e490c7aaa1ea'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:58.682973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1341/2713] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:18:58 2022] DELETE /identity/v3/users/2fbbbb374d284e9b9ad2e490c7aaa1ea => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:18:58.687400 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.687682 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.687955 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.709693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:58.713628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:58.776226 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1373/2714] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:18:58.785780 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:58.787908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:58.843131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:18:58.855128 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e20493-dfa1-4fb0-9b90-69bf989e6ebc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:58.858794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327b7bd-5ca1-4426-9da7-91e74939dc62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:58.859384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327b7bd-5ca1-4426-9da7-91e74939dc62 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:58.859669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327b7bd-5ca1-4426-9da7-91e74939dc62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:58.859948 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327b7bd-5ca1-4426-9da7-91e74939dc62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:58.869173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327b7bd-5ca1-4426-9da7-91e74939dc62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:58.870644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327b7bd-5ca1-4426-9da7-91e74939dc62 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:58.880238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1342/2715] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:18:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:18:59.515868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:18:59.517543 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:59.532767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b788f9-21ab-4f9a-9c35-71904e3b4ce6 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:18:59.541518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:18:59.542791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:59.543271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:59.543755 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] PATH_INFO: `/v3/projects/2ecb2b23305249c7aa6c328ef0dafc99` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:59.546719 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ecb2b23305249c7aa6c328ef0dafc99)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:18:59.552840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:18:59.719166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ecb2b23305249c7aa6c328ef0dafc99', 'request_id': 'req-0469808d-11d4-45bc-8f7a-f155b4e32faf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:59.867628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ecb2b23305249c7aa6c328ef0dafc99', 'request_id': 'req-0469808d-11d4-45bc-8f7a-f155b4e32faf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:59.875650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ecb2b23305249c7aa6c328ef0dafc99', 'request_id': 'req-0469808d-11d4-45bc-8f7a-f155b4e32faf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:59.890112 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] The token cache is being invalidate because project 2ecb2b23305249c7aa6c328ef0dafc99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:18:59.890572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:59.891257 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0469808d-11d4-45bc-8f7a-f155b4e32faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:18:59.893207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1374/2716] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:18:59 2022] DELETE /identity/v3/projects/2ecb2b23305249c7aa6c328ef0dafc99 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:18:59.899928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7e955-db62-4199-b755-68786515595c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:18:59.899928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7e955-db62-4199-b755-68786515595c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:18:59.899928 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda7e955-db62-4199-b755-68786515595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:18:59.921620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dda7e955-db62-4199-b755-68786515595c None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:18:59.922933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dda7e955-db62-4199-b755-68786515595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:18:59.994811 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1343/2717] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:00.005431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.005717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.006109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.027768 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:00.028983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:00.037602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1375/2718] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:00.041813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7e955-db62-4199-b755-68786515595c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.052366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda7e955-db62-4199-b755-68786515595c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.057455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.058033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.058317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.058592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.059240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:00.059541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.061253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.075690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1344/2719] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:00.080647 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.091769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6e17d-194b-4ad4-8ed0-f1e5c4bfed95 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.096989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.097670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.098011 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.109291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.109291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.109291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.226820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1376/2720] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:00 2022] POST /identity/v3/projects => generated 403 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:00.228767 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.240480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babdf0e9-4023-4cd8-96a9-1870462f0b10 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.316458 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.317617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.318734 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.319028 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.319689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.321718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.401221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1345/2721] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:00 2022] POST /identity/v3/users => generated 302 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:00.405701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.418850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61cb391-e80e-41c5-8667-bf016f3d1f2f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.423616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.424228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.424554 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.424836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.425556 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.427428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.437966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1377/2722] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:00.447973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.460832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf6f382-49f9-456a-8a5a-bb6d165a2272 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.471034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b561ab-54dd-4fe5-8967-82ac04f3aca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.472204 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b561ab-54dd-4fe5-8967-82ac04f3aca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.472559 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b561ab-54dd-4fe5-8967-82ac04f3aca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.472880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b561ab-54dd-4fe5-8967-82ac04f3aca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.473596 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b561ab-54dd-4fe5-8967-82ac04f3aca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.475584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b561ab-54dd-4fe5-8967-82ac04f3aca9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.495491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1346/2723] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:00.499829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.519568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43569144-858e-499a-9131-fb8a378539d0 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.519568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f501-e294-4017-a634-839209124049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.519568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f501-e294-4017-a634-839209124049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.519568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f501-e294-4017-a634-839209124049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.521313 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460f501-e294-4017-a634-839209124049 admin admin] PATH_INFO: `/v3/projects/0cd7f8f937eb4a2ca28a16c8aa606863/users/0f0394b480c9488893d9798cb0b838a7/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.536775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460f501-e294-4017-a634-839209124049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd7f8f937eb4a2ca28a16c8aa606863, user_id=0f0394b480c9488893d9798cb0b838a7, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.539918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460f501-e294-4017-a634-839209124049 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.686475 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:00.692593 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1378/2724] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:00 2022] PUT /identity/v3/projects/0cd7f8f937eb4a2ca28a16c8aa606863/users/0f0394b480c9488893d9798cb0b838a7/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:00.703895 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d345a-8817-400d-9866-798abe60911c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.704260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d345a-8817-400d-9866-798abe60911c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.704541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d345a-8817-400d-9866-798abe60911c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.734933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5a8d345a-8817-400d-9866-798abe60911c None None] MFA Rules not processed for user `0f0394b480c9488893d9798cb0b838a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:00.736193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a8d345a-8817-400d-9866-798abe60911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:00.783317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:00.816255 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1347/2725] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:00.821532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f501-e294-4017-a634-839209124049 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.901688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460f501-e294-4017-a634-839209124049 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.907824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.908583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.908893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:00.909403 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:00.910101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:00.910945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d345a-8817-400d-9866-798abe60911c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:00.913276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:00.913448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a8d345a-8817-400d-9866-798abe60911c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:00.995260 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d345a-8817-400d-9866-798abe60911c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:00.998869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:00.999479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:00.999759 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.000035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.001936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.020402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1379/2726] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:00 2022] POST /identity/v3/projects => generated 405 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:01.021428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.076313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.077793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.087841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aae7ce-4097-4879-8b8c-d19608eb213b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.088098 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1348/2727] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:01.092836 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.093287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.093573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.093868 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.094503 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.096409 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.182376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1380/2728] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:01 2022] POST /identity/v3/users => generated 309 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:01.186597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.198492 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f0603-785f-45d1-b384-14d3f1c22615 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.203444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.204031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.204342 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.204617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.205324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.206957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.217634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1349/2729] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:01.222152 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.232443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f83ee8-e696-4d33-b7a1-1bb58f3f2c51 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.237139 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.237812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.241787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.241787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] PATH_INFO: `/v3/projects/5358ccc9a0584a9dafbeea736f930807/users/9fdd9ccef5914c1faf12f6cd676d9bbf/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.254701 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5358ccc9a0584a9dafbeea736f930807, user_id=9fdd9ccef5914c1faf12f6cd676d9bbf, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.257469 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.393917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1381/2730] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:01 2022] PUT /identity/v3/projects/5358ccc9a0584a9dafbeea736f930807/users/9fdd9ccef5914c1faf12f6cd676d9bbf/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:01.397933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.409057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d604a718-d5a6-443c-8209-b0bbb49d3397 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.483869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ed02d-c6ce-4062-aca8-3ba15748e63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.484508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ed02d-c6ce-4062-aca8-3ba15748e63c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.484793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ed02d-c6ce-4062-aca8-3ba15748e63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.485079 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ed02d-c6ce-4062-aca8-3ba15748e63c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.485706 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ed02d-c6ce-4062-aca8-3ba15748e63c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.490511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ed02d-c6ce-4062-aca8-3ba15748e63c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.503889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1350/2731] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:01.508852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.519443 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d832df06-d0f7-427c-aff7-d96e7b156ad2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.525342 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.526045 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.526324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.526616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] PATH_INFO: `/v3/projects/5358ccc9a0584a9dafbeea736f930807/users/9fdd9ccef5914c1faf12f6cd676d9bbf/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.528611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5358ccc9a0584a9dafbeea736f930807, user_id=9fdd9ccef5914c1faf12f6cd676d9bbf, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.531287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.543331 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1382/2732] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:01 2022] PUT /identity/v3/projects/5358ccc9a0584a9dafbeea736f930807/users/9fdd9ccef5914c1faf12f6cd676d9bbf/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:01.555940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.555940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.555940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.585224 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] MFA Rules not processed for user `9fdd9ccef5914c1faf12f6cd676d9bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:01.589594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.657364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1351/2733] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:01 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:01.672130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:01.674394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:01.742885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f79260d-1652-439d-bd3e-0549607446b7 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:01.747402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727f930-a893-456a-855a-2e760e2fe0a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:01.748016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727f930-a893-456a-855a-2e760e2fe0a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:01.748308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727f930-a893-456a-855a-2e760e2fe0a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:01.748586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727f930-a893-456a-855a-2e760e2fe0a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:01.759082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727f930-a893-456a-855a-2e760e2fe0a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:01.760527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727f930-a893-456a-855a-2e760e2fe0a3 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:01.770690 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1383/2734] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:01 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:02.013594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:02.076885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad70faf-c41c-4461-926a-f95912340f16 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:02.081426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:02.082086 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:02.082397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:02.082711 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] PATH_INFO: `/v3/users/0f0394b480c9488893d9798cb0b838a7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:02.083974 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f0394b480c9488893d9798cb0b838a7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:02.085920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:02.099351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:02.315703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f0394b480c9488893d9798cb0b838a7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.323242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f0394b480c9488893d9798cb0b838a7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.340453 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:02.391880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f0394b480c9488893d9798cb0b838a7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.393344 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f0394b480c9488893d9798cb0b838a7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.406766 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f0394b480c9488893d9798cb0b838a7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:02.414185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1352/2735] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:02 2022] DELETE /identity/v3/users/0f0394b480c9488893d9798cb0b838a7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:02.418756 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:02.418978 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:02.419247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:02.438191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:02.439375 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:02.499471 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1384/2736] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:02.509844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:02.511476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:02.548526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:02.601333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b31a5-5bec-4497-b5ad-36451333cf2b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:02.605111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:02.605741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:02.606038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:02.606317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:02.616103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:02.617576 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:02.627448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1353/2737] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:02.936947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:02.938483 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:03.008206 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0584231-2d71-425d-8269-05cf278e87a3 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:03.012459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:03.013073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:03.013356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:03.013630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:03.015482 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:03.083102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:03.084718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:03.094649 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1385/2738] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:03.211613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:03.212986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:03.228249 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24c76e4-1610-4516-841f-b1fb96da05b6 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:03.233096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:03.233749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:03.234062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:03.234398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] PATH_INFO: `/v3/projects/0cd7f8f937eb4a2ca28a16c8aa606863` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:03.235568 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd7f8f937eb4a2ca28a16c8aa606863)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:03.237588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:03.371406 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd7f8f937eb4a2ca28a16c8aa606863', 'request_id': 'req-d6aed1c5-f34f-4eca-b144-3f3914e0039f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.377500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd7f8f937eb4a2ca28a16c8aa606863', 'request_id': 'req-d6aed1c5-f34f-4eca-b144-3f3914e0039f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.384326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd7f8f937eb4a2ca28a16c8aa606863', 'request_id': 'req-d6aed1c5-f34f-4eca-b144-3f3914e0039f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.484303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] The token cache is being invalidate because project 0cd7f8f937eb4a2ca28a16c8aa606863 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:03.484556 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.485228 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.486775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1354/2739] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:03 2022] DELETE /identity/v3/projects/0cd7f8f937eb4a2ca28a16c8aa606863 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:03.491425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:03.502281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89f8e1-1747-4684-b9d0-565116b3dc34 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:03.532250 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=100 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:03.560406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:03.561039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:03.561324 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:03.561602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] PATH_INFO: `/v3/users/9fdd9ccef5914c1faf12f6cd676d9bbf` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:03.562769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fdd9ccef5914c1faf12f6cd676d9bbf)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:03.564714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:03.790767 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fdd9ccef5914c1faf12f6cd676d9bbf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.798087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fdd9ccef5914c1faf12f6cd676d9bbf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.798830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fdd9ccef5914c1faf12f6cd676d9bbf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.799492 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fdd9ccef5914c1faf12f6cd676d9bbf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.812531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-603dd33c-8f2f-48ae-83f2-1c59415fa218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fdd9ccef5914c1faf12f6cd676d9bbf'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:03.883659 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1386/2740] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:03 2022] DELETE /identity/v3/users/9fdd9ccef5914c1faf12f6cd676d9bbf => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:04.274699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.276176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.294073 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aed1c5-f34f-4eca-b144-3f3914e0039f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.361752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.362861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.363392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.364404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] PATH_INFO: `/v3/projects/5358ccc9a0584a9dafbeea736f930807` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.366393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5358ccc9a0584a9dafbeea736f930807)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.369293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.517415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:04.527762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5358ccc9a0584a9dafbeea736f930807', 'request_id': 'req-f3228d92-7b24-441d-96c4-03526ef69759'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.534105 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5358ccc9a0584a9dafbeea736f930807', 'request_id': 'req-f3228d92-7b24-441d-96c4-03526ef69759'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.541035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5358ccc9a0584a9dafbeea736f930807', 'request_id': 'req-f3228d92-7b24-441d-96c4-03526ef69759'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.699636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] The token cache is being invalidate because project 5358ccc9a0584a9dafbeea736f930807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:04.699856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.700847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f3228d92-7b24-441d-96c4-03526ef69759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:04.702705 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1355/2741] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:04 2022] DELETE /identity/v3/projects/5358ccc9a0584a9dafbeea736f930807 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:04.709032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.709567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.709900 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.732258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:04.733481 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.797269 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1387/2742] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:04.807021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.807316 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.807599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.828635 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:04.830131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:04.838916 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1356/2743] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:04.843957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.854693 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367fdd21-21da-476e-a4ca-7d6395caf854 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.859787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.860383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.860665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.860946 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.861594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:04.861950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.863785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.875742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1388/2744] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:04.880323 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:04.889621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ba11b-ba42-4193-9fba-dcb7c8a84930 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:04.894401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:04.896046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:04.896364 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:04.896645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:04.897283 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:04.899363 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:04.999185 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1357/2745] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:04 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:05.003570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.014490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56576693-3b80-457a-bf12-6be4d1067725 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.048670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:05.072115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.072764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.073048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.076121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.076121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.076121 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.187146 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1389/2746] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:05 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:05.192088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.202920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0329f301-11ea-4148-b2ed-25096f27991e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.208525 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.209123 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.209397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.209665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.210336 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.211903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.224154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1358/2747] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:05.228346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.239093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7068c-d9dc-447f-b92b-ee05b221ddc5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.244220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829ba262-8043-4df7-85d8-c09db8a40c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.244834 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ba262-8043-4df7-85d8-c09db8a40c05 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.246145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ba262-8043-4df7-85d8-c09db8a40c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.246145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829ba262-8043-4df7-85d8-c09db8a40c05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.246145 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ba262-8043-4df7-85d8-c09db8a40c05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.247695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829ba262-8043-4df7-85d8-c09db8a40c05 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.262650 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1390/2748] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:05.263820 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.274548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4893d1-a22b-4958-bd3a-2a3d91e44e3c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.279200 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.279848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.280132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.280410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] PATH_INFO: `/v3/projects/9151ec01210b4868bdace4ab81ecc20a/users/500555c62027444e89e561de81b4c87b/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.295545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9151ec01210b4868bdace4ab81ecc20a, user_id=500555c62027444e89e561de81b4c87b, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.298222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.380910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1359/2749] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:05 2022] PUT /identity/v3/projects/9151ec01210b4868bdace4ab81ecc20a/users/500555c62027444e89e561de81b4c87b/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:05.391298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.391586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.391862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.424940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] MFA Rules not processed for user `500555c62027444e89e561de81b4c87b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:05.426961 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:05.493784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1391/2750] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:05.498426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.562159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298d154-dbab-43e7-b955-ec1bc963a74d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.567800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.568407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.568682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.568952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.569595 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.571589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.669408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1360/2751] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:05 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:05.673565 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.685074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da373aa-45f7-4deb-86f7-5c047f3ac049 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.743402 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.744127 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.744314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.744592 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.745255 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.747841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.851490 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1392/2752] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:05 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:05.855812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.867133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ab34e-df78-4231-9fc5-c0dc87c3d034 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.871894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.872591 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.872890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.873183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.873881 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.875486 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:05.886409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1361/2753] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:05.890639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:05.900862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd98d4-bae6-4b99-ba72-d872fa39f7c2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:05.905459 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:05.906147 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:05.906431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:05.906709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] PATH_INFO: `/v3/projects/c25e05b8a3b346fc807e7164a25bcb58/users/b1e92359d24e423a80a27811b1467f84/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:05.922698 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25e05b8a3b346fc807e7164a25bcb58, user_id=b1e92359d24e423a80a27811b1467f84, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:05.925431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.009882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1393/2754] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:05 2022] PUT /identity/v3/projects/c25e05b8a3b346fc807e7164a25bcb58/users/b1e92359d24e423a80a27811b1467f84/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:06.013950 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.024951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd65b23-2c73-4fd9-aad4-d5103eb03ae3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.055060 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:06.081645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0162043c-a183-4a3b-a8c5-aa7f2c93d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.083048 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162043c-a183-4a3b-a8c5-aa7f2c93d7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.083329 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162043c-a183-4a3b-a8c5-aa7f2c93d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.083603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0162043c-a183-4a3b-a8c5-aa7f2c93d7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.084242 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162043c-a183-4a3b-a8c5-aa7f2c93d7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.085884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0162043c-a183-4a3b-a8c5-aa7f2c93d7a8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.097068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1362/2755] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:06 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:06.102363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.113790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff4056-df4b-4ea8-9d9f-6cb7bc8a511b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.118920 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.119579 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.119851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.120123 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] PATH_INFO: `/v3/projects/c25e05b8a3b346fc807e7164a25bcb58/users/b1e92359d24e423a80a27811b1467f84/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.122268 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c25e05b8a3b346fc807e7164a25bcb58, user_id=b1e92359d24e423a80a27811b1467f84, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.124983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.131422 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:06.137316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1394/2756] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:06 2022] PUT /identity/v3/projects/c25e05b8a3b346fc807e7164a25bcb58/users/b1e92359d24e423a80a27811b1467f84/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:06.148695 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.148988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.149264 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.178471 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] MFA Rules not processed for user `b1e92359d24e423a80a27811b1467f84`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:06.179672 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.248785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1363/2757] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:06.259999 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.261664 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.342891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7fd7e2-abe7-47ba-b0d2-a6c6ef4be81d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.346569 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.347181 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.347468 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.348265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.358612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.360050 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.369913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1395/2758] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:06.598016 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:06.602382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:06.621934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85e6fc1-8320-4060-b230-f6735d9e7030 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:06.625617 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6eff4c-b7e9-4e70-bd51-72c84b29ba08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:06.626250 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6eff4c-b7e9-4e70-bd51-72c84b29ba08 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:06.626529 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6eff4c-b7e9-4e70-bd51-72c84b29ba08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:06.626801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6eff4c-b7e9-4e70-bd51-72c84b29ba08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:06.691068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6eff4c-b7e9-4e70-bd51-72c84b29ba08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:06.693469 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6eff4c-b7e9-4e70-bd51-72c84b29ba08 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:06.702922 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1364/2759] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:11.129938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:11.131377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:11.202591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48348fc7-f335-445d-9ed5-3814e046d008 service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:11.208108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:11.208919 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:11.209388 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:11.209770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] PATH_INFO: `/v3/users/684a8f7a5cda4a1688cc75fc0ef0d788` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:11.210998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=684a8f7a5cda4a1688cc75fc0ef0d788)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:11.213334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:11.405551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '684a8f7a5cda4a1688cc75fc0ef0d788'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:11.413980 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '684a8f7a5cda4a1688cc75fc0ef0d788'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:11.414710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '684a8f7a5cda4a1688cc75fc0ef0d788'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:11.415354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '684a8f7a5cda4a1688cc75fc0ef0d788'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:11.428904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '684a8f7a5cda4a1688cc75fc0ef0d788'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:11.503184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:11.508222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1396/2760] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:11 2022] DELETE /identity/v3/users/684a8f7a5cda4a1688cc75fc0ef0d788 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:11.512890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:11.513393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:11.513493 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:11.532325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:11.533546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:11.581897 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:11.593721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1365/2761] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:11.603684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:11.605094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:11.664607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f211c9-efc7-413e-83e5-820c2c98e72f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:11.668427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:11.669014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:11.669295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:11.669570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:11.679985 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:11.681416 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:11.691036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1397/2762] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:12.120183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:12.121904 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:12.134167 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29035a-1908-42df-9910-1d2ce8bc06bb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:12.138753 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:12.139400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:12.139686 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:12.139965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] PATH_INFO: `/v3/projects/5b93bc70d91e4761bc3cad29d8061825` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:12.141094 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b93bc70d91e4761bc3cad29d8061825)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:12.143108 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:12.261193 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b93bc70d91e4761bc3cad29d8061825', 'request_id': 'req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.267834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b93bc70d91e4761bc3cad29d8061825', 'request_id': 'req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.275212 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b93bc70d91e4761bc3cad29d8061825', 'request_id': 'req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.381096 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] The token cache is being invalidate because project 5b93bc70d91e4761bc3cad29d8061825 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:12.381387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.382057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.383551 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1366/2763] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:12 2022] DELETE /identity/v3/projects/5b93bc70d91e4761bc3cad29d8061825 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:12.389114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:12.400020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579aad3-59ad-4136-9fc9-85dd061e181a service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:12.460922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:12.461567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:12.461885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:12.462166 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] PATH_INFO: `/v3/users/4bffad21692a4cd1bf17ab0564fa520d` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:12.463306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bffad21692a4cd1bf17ab0564fa520d)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:12.465230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:12.680527 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bffad21692a4cd1bf17ab0564fa520d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.687570 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bffad21692a4cd1bf17ab0564fa520d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.688333 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bffad21692a4cd1bf17ab0564fa520d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.688937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bffad21692a4cd1bf17ab0564fa520d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.701390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bffad21692a4cd1bf17ab0564fa520d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:12.804981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1398/2764] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:12 2022] DELETE /identity/v3/users/4bffad21692a4cd1bf17ab0564fa520d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:13.238518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:13.240077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:13.253393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462f5f9e-fe3d-46e0-b10a-aa4260811bf0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:13.313348 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:13.314026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:13.314313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:13.314594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] PATH_INFO: `/v3/projects/a5b647fedc864215a8bcf90a1738ef62` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:13.316052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5b647fedc864215a8bcf90a1738ef62)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:13.318727 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:13.482871 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5b647fedc864215a8bcf90a1738ef62', 'request_id': 'req-928b37ab-a016-45c1-8bad-43bfcf9625b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.489761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5b647fedc864215a8bcf90a1738ef62', 'request_id': 'req-928b37ab-a016-45c1-8bad-43bfcf9625b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.495506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5b647fedc864215a8bcf90a1738ef62', 'request_id': 'req-928b37ab-a016-45c1-8bad-43bfcf9625b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.598360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] The token cache is being invalidate because project a5b647fedc864215a8bcf90a1738ef62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:13.598659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.599331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.600874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1367/2765] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:13 2022] DELETE /identity/v3/projects/a5b647fedc864215a8bcf90a1738ef62 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:13.604992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:13.606495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:13.619677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8307ef-3b30-4c7f-96cc-41cc6750661d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:13.638538 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:13.678034 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:13.678671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:13.678957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:13.679234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] PATH_INFO: `/v3/users/f50ac7590c1a4ec5b23ff3673fed31e7` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:13.680393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f50ac7590c1a4ec5b23ff3673fed31e7)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:13.682365 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:13.853854 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f50ac7590c1a4ec5b23ff3673fed31e7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.861526 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f50ac7590c1a4ec5b23ff3673fed31e7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.862360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f50ac7590c1a4ec5b23ff3673fed31e7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.863833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f50ac7590c1a4ec5b23ff3673fed31e7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.876654 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f50ac7590c1a4ec5b23ff3673fed31e7'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:13.970521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1399/2766] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:13 2022] DELETE /identity/v3/users/f50ac7590c1a4ec5b23ff3673fed31e7 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:14.412502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:14.414282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:14.429609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928b37ab-a016-45c1-8bad-43bfcf9625b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:14.433420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:14.491743 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:14.492382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.492671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.492970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] PATH_INFO: `/v3/projects/d8a13e8edcfe4f119e4033bc117a0924` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.494161 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a13e8edcfe4f119e4033bc117a0924)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:14.496106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:14.564217 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:14.565665 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:14.579195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f83e19-e939-4ce1-98ee-cea14131d552 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:14.584046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:14.584675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.584952 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.585227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] PATH_INFO: `/v3/users/f3ca11c79c1b447494718d70a8a8cfa6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.586346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ca11c79c1b447494718d70a8a8cfa6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:14.588287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:14.608984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a13e8edcfe4f119e4033bc117a0924', 'request_id': 'req-a0c46ba1-18d6-41c8-9588-a73c23986b01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.616043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a13e8edcfe4f119e4033bc117a0924', 'request_id': 'req-a0c46ba1-18d6-41c8-9588-a73c23986b01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.621817 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a13e8edcfe4f119e4033bc117a0924', 'request_id': 'req-a0c46ba1-18d6-41c8-9588-a73c23986b01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.735644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] The token cache is being invalidate because project d8a13e8edcfe4f119e4033bc117a0924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:14.735891 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.736481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a0c46ba1-18d6-41c8-9588-a73c23986b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.738703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1368/2767] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:14 2022] DELETE /identity/v3/projects/d8a13e8edcfe4f119e4033bc117a0924 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:14.743537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfa181-55e2-480e-90ff-6fe341af1106 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.743827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfa181-55e2-480e-90ff-6fe341af1106 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.744136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfa181-55e2-480e-90ff-6fe341af1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.764407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c2cfa181-55e2-480e-90ff-6fe341af1106 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:14.765638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c2cfa181-55e2-480e-90ff-6fe341af1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:14.826736 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1369/2768] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:14.829197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ca11c79c1b447494718d70a8a8cfa6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.834813 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.835136 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.835420 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.837308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ca11c79c1b447494718d70a8a8cfa6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.838040 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ca11c79c1b447494718d70a8a8cfa6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.838652 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ca11c79c1b447494718d70a8a8cfa6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.852529 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-730f8726-1dd6-434a-9d25-9dd32584ec50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ca11c79c1b447494718d70a8a8cfa6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:14.858110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:14.859460 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:14.868232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1370/2769] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:14.872575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:14.883007 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aa8ca7-ca7e-4db9-8bd0-56f9d7074541 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:14.888102 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:14.888940 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.889164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.889450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.890134 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:14.890444 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:14.892150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:14.908000 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1371/2770] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:19:14.909609 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:14.920394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ea3e0-5b17-4f4c-b967-a528762747d7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:14.925360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:14.926724 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.927014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.927282 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.927951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:14.930012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:14.969993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1400/2771] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:14 2022] DELETE /identity/v3/users/f3ca11c79c1b447494718d70a8a8cfa6 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:14.974932 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:14.975227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:14.975516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:14.996109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:14.997319 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:15.006691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:15.020834 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1372/2772] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:14 2022] POST /identity/v3/projects => generated 369 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:15.022319 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.038880 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:15.070191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1401/2773] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:15.081709 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.083391 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:15.098971 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d861ca9-efcd-4748-ac17-748cea81ea4b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.103839 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.104453 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.104735 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.105014 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.105699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.107642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.143169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effc23b3-9be0-4123-ba92-8c4c0bd93a73 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.190810 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.191975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.192256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.192530 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.202256 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.204001 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.214990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1402/2774] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:15.243636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1373/2775] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:15 2022] POST /identity/v3/users => generated 285 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:15.244722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.256116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30231f44-99bb-4ed4-b2a6-8fb770d10c7d service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.260820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.261420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.261703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.262009 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.262648 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.264238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.277692 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1403/2776] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:15.279026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.289920 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee8c6e5-c9c5-4c67-9fb9-a4d6f1bf0193 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.294943 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87503984-63d6-44f0-9550-906e06a230f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.295773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503984-63d6-44f0-9550-906e06a230f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.296062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503984-63d6-44f0-9550-906e06a230f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.296369 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87503984-63d6-44f0-9550-906e06a230f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.296985 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503984-63d6-44f0-9550-906e06a230f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.298690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87503984-63d6-44f0-9550-906e06a230f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.309533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1374/2777] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:15.313691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.324408 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656df185-4ba3-4a5e-af6d-21a2ebb48513 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.330008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.330669 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.330951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.331227 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] PATH_INFO: `/v3/projects/6d66050655494b9880890460ef7ee8b5/users/051c8050236a4882b5c1187a8f403981/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.346507 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d66050655494b9880890460ef7ee8b5, user_id=051c8050236a4882b5c1187a8f403981, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.349880 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.456014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1404/2778] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:15 2022] PUT /identity/v3/projects/6d66050655494b9880890460ef7ee8b5/users/051c8050236a4882b5c1187a8f403981/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:15.468118 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.468625 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.468748 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.506041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] MFA Rules not processed for user `051c8050236a4882b5c1187a8f403981`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:15.507437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:15.588198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1375/2779] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:15.600008 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.603546 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:15.648151 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:15.663703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa3c84-27ff-446e-909f-d2117237a3d6 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.667466 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829d0a0d-a560-4317-99a7-bdcc09789e1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0f8fc283cce4eff8126074e2a2c39fa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.671794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d0a0d-a560-4317-99a7-bdcc09789e1c service glance] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.671794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d0a0d-a560-4317-99a7-bdcc09789e1c service glance] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.671794 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d0a0d-a560-4317-99a7-bdcc09789e1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.677213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d0a0d-a560-4317-99a7-bdcc09789e1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.678670 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829d0a0d-a560-4317-99a7-bdcc09789e1c service glance] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.687442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1405/2780] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:15.697431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:15.699074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:15.786131 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397b9be-de44-4fbc-98fa-4fc2f6df0fe3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:15.791141 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:15.791783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:15.792068 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:15.792353 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] PATH_INFO: `/v3/projects/e328fc06dc19416a889a1f08aef03943` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:15.793608 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e328fc06dc19416a889a1f08aef03943)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:15.796676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:15.811782 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:15.915424 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e328fc06dc19416a889a1f08aef03943', 'request_id': 'req-190f8c70-95df-4df4-849a-2a2b2f4a957c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:15.922314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e328fc06dc19416a889a1f08aef03943', 'request_id': 'req-190f8c70-95df-4df4-849a-2a2b2f4a957c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:15.930674 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e328fc06dc19416a889a1f08aef03943', 'request_id': 'req-190f8c70-95df-4df4-849a-2a2b2f4a957c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:16.025534 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] The token cache is being invalidate because project e328fc06dc19416a889a1f08aef03943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:16.025773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:16.026416 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-190f8c70-95df-4df4-849a-2a2b2f4a957c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:16.027918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1376/2781] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:15 2022] DELETE /identity/v3/projects/e328fc06dc19416a889a1f08aef03943 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:16.033074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.033595 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.033798 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.055585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:16.056800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:16.118586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1406/2782] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:16.127541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.127864 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.128237 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.150066 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:16.151279 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:16.159456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1377/2783] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:16.164841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:16.174688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0cb2b9-1d87-4cf1-ab5b-90fc17f84fb8 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:16.180184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:16.180783 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.181063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.181337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.182021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:16.182326 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:16.184082 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:16.196129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1407/2784] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:16.200606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:16.210081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4667539b-6379-46b7-ad82-5c6de0018687 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:16.214689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:16.215346 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.215631 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.215903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.216524 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:16.218554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:16.354959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1378/2785] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:16 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:16.359251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:16.370047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49377c04-7bdd-4516-8c0c-7e9eb0e42fba admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:16.429996 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:16.430600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.430936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.431220 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.431875 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:16.433764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:16.588263 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1408/2786] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:16 2022] POST /identity/v3/users => generated 288 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:16.593226 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:16.607478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029e49e-0e21-4537-a57d-ef915c8b86cc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:16.613021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:16.626429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.626429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.626429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.626429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:16.626429 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:16.630520 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1379/2787] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:16.641666 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:16.652451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde73ffb-ee35-4ceb-9564-1c46cde45a3c admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:16.657122 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d20d2-d230-4015-869c-41ab03296e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:16.657712 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d20d2-d230-4015-869c-41ab03296e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.658018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d20d2-d230-4015-869c-41ab03296e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.658287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d20d2-d230-4015-869c-41ab03296e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.658949 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d20d2-d230-4015-869c-41ab03296e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:16.661860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d20d2-d230-4015-869c-41ab03296e35 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:16.673079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1409/2788] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:16.677702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:16.688413 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d064cb-71a6-481b-9016-ad30728f68e9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:16.693170 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:16.693909 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.694238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.694513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] PATH_INFO: `/v3/projects/df5d9d59d2a246298cb28e971b9b5eca/users/4905d31c7ab9450e99b6e2f69dd42431/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.714046 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5d9d59d2a246298cb28e971b9b5eca, user_id=4905d31c7ab9450e99b6e2f69dd42431, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:16.716699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:16.878243 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1380/2789] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:16 2022] PUT /identity/v3/projects/df5d9d59d2a246298cb28e971b9b5eca/users/4905d31c7ab9450e99b6e2f69dd42431/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:16.889764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:16.890063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:16.890340 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:16.920837 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] MFA Rules not processed for user `4905d31c7ab9450e99b6e2f69dd42431`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:16.922253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:16.964209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:16.996886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1410/2790] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:17.012384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:17.012384 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:17.093439 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bba72f-b75e-4c27-b808-d5b697d0ef5c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:17.097311 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35517b3a-9f35-407c-88d7-6eb2468c0f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:17.097992 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517b3a-9f35-407c-88d7-6eb2468c0f6a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:17.098274 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517b3a-9f35-407c-88d7-6eb2468c0f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:17.098548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35517b3a-9f35-407c-88d7-6eb2468c0f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:17.107938 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35517b3a-9f35-407c-88d7-6eb2468c0f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:17.109558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35517b3a-9f35-407c-88d7-6eb2468c0f6a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:17.119309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1381/2791] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:19.926064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:19.927787 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:19.998184 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6dc9f-5932-4b86-816a-cfcc705d6809 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:20.002537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:20.003153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:20.003942 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:20.004212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] PATH_INFO: `/v3/users/500555c62027444e89e561de81b4c87b` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:20.005406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=500555c62027444e89e561de81b4c87b)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:20.007353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:20.406370 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500555c62027444e89e561de81b4c87b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:20.414578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500555c62027444e89e561de81b4c87b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:20.415444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500555c62027444e89e561de81b4c87b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:20.416231 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500555c62027444e89e561de81b4c87b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:20.429135 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500555c62027444e89e561de81b4c87b'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:20.633201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1411/2792] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:19 2022] DELETE /identity/v3/users/500555c62027444e89e561de81b4c87b => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:20.637681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:20.638241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:20.638368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:20.659268 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:20.661039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:20.696594 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:20.733261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1382/2793] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:20.746741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:20.748245 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:20.811805 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106186bd-1610-481d-8bcf-23492a8b48ef admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:20.815792 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:20.816463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:20.816806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:20.817197 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:20.827600 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:20.828358 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:20.832567 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:20.838773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1412/2794] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:21.303190 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:21.305179 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:21.324863 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda7653e-0028-4380-aa0e-9116f238f556 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:21.336816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:21.337704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:21.339987 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:21.340277 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] PATH_INFO: `/v3/projects/9151ec01210b4868bdace4ab81ecc20a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:21.341522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9151ec01210b4868bdace4ab81ecc20a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:21.345503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:21.502720 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9151ec01210b4868bdace4ab81ecc20a', 'request_id': 'req-18117a3f-c0b7-4d72-966b-52d96a3cef07'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.509569 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9151ec01210b4868bdace4ab81ecc20a', 'request_id': 'req-18117a3f-c0b7-4d72-966b-52d96a3cef07'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.515221 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9151ec01210b4868bdace4ab81ecc20a', 'request_id': 'req-18117a3f-c0b7-4d72-966b-52d96a3cef07'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.640873 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] The token cache is being invalidate because project 9151ec01210b4868bdace4ab81ecc20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:21.641552 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.641851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.643409 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1383/2795] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:21 2022] DELETE /identity/v3/projects/9151ec01210b4868bdace4ab81ecc20a => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:21.648036 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:21.658153 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e9724-9073-4dfc-a746-6b064cd6d271 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:21.717276 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:21.717945 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:21.718232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:21.718511 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] PATH_INFO: `/v3/users/b1e92359d24e423a80a27811b1467f84` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:21.719643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e92359d24e423a80a27811b1467f84)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:21.721616 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:21.961246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e92359d24e423a80a27811b1467f84'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.968389 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e92359d24e423a80a27811b1467f84'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.969102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e92359d24e423a80a27811b1467f84'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.969722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e92359d24e423a80a27811b1467f84'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:21.982881 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-6a9ff743-7f6c-46cc-be9f-fd2d47749be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e92359d24e423a80a27811b1467f84'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:22.049211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1413/2796] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:21 2022] DELETE /identity/v3/users/b1e92359d24e423a80a27811b1467f84 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:22.619324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:22.620847 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:22.640859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18117a3f-c0b7-4d72-966b-52d96a3cef07 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:22.705292 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:22.706042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:22.706244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:22.707062 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] PATH_INFO: `/v3/projects/c25e05b8a3b346fc807e7164a25bcb58` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:22.708303 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c25e05b8a3b346fc807e7164a25bcb58)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:22.710244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:22.828479 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c25e05b8a3b346fc807e7164a25bcb58', 'request_id': 'req-ce8133db-0747-42ad-afef-1c07a9eba38c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:22.835957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c25e05b8a3b346fc807e7164a25bcb58', 'request_id': 'req-ce8133db-0747-42ad-afef-1c07a9eba38c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:22.840033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:22.842174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c25e05b8a3b346fc807e7164a25bcb58', 'request_id': 'req-ce8133db-0747-42ad-afef-1c07a9eba38c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:22.966542 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] The token cache is being invalidate because project c25e05b8a3b346fc807e7164a25bcb58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:22.966838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:22.967574 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ce8133db-0747-42ad-afef-1c07a9eba38c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:22.969541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1384/2797] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:22 2022] DELETE /identity/v3/projects/c25e05b8a3b346fc807e7164a25bcb58 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:22.974399 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:22.974930 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:22.975039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:22.996821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:22.998188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:23.070266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1414/2798] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:23.078297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.078589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.078869 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.100145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:23.101526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:23.113389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1385/2799] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:23.114398 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.124671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65a029-5ca9-43ac-8fda-fee3d18a5aeb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.130164 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.130770 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.131052 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.131328 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.131976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:23.132315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.133977 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:23.153077 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1415/2800] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:19:23.154269 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.164074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de3fed-5e28-4854-8454-ae11ba59c07e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.168693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.169296 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.169571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.169874 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.170503 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.172378 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:23.277523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1386/2801] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:23 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:23.282947 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.292150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:23.296641 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e1dc76-ecf8-4912-9768-7820c8aba187 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.358113 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.358633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.358913 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.359189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.359825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.361771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:23.512911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1416/2802] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:23 2022] POST /identity/v3/users => generated 294 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:23.517155 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.528835 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b1563a-3372-450d-9b54-2ee8b15e9ef3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.532958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.533793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.534070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.534354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.535074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.536942 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:23.551034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1387/2803] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:23.554230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.567341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a3ece9-29e5-47bc-9aa2-5848775b3b77 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.575480 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9200ce0c-d830-4412-ad41-1a550f11cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.576450 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200ce0c-d830-4412-ad41-1a550f11cc39 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.576917 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200ce0c-d830-4412-ad41-1a550f11cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.577377 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9200ce0c-d830-4412-ad41-1a550f11cc39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.578518 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200ce0c-d830-4412-ad41-1a550f11cc39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.581393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9200ce0c-d830-4412-ad41-1a550f11cc39 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:23.593521 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1417/2804] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:23.597615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.607860 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effabbf4-e814-4328-a73b-bc9fc0f2068f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.612500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.613903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.614320 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.614666 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] PATH_INFO: `/v3/projects/0430d4f343034016b189217c9d615ad0/users/1ff33eb661884f19831f4cacfe54047b/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.635034 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0430d4f343034016b189217c9d615ad0, user_id=1ff33eb661884f19831f4cacfe54047b, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.638070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:23.730689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1388/2805] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:23 2022] PUT /identity/v3/projects/0430d4f343034016b189217c9d615ad0/users/1ff33eb661884f19831f4cacfe54047b/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:23.742351 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.742721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.743064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.773860 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] MFA Rules not processed for user `1ff33eb661884f19831f4cacfe54047b`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:23.775095 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:23.856406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1418/2806] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:23.860967 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:23.938952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d371838-17fb-4e74-9297-b865d04817a1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:23.947721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:23.947721 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:23.948877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:23.949154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:23.949832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:23.951746 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.041645 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1389/2807] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:23 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:24.046019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:24.057831 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc4b510-c44a-4eb4-93bd-25017fabc884 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:24.127816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:24.128378 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.128777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.129280 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.130019 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:24.133022 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.290284 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1419/2808] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:24 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:24.294262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:24.324481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c20d72-ea33-4525-9fc9-d17f074a2359 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:24.324481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:24.324481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.324481 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.324986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.324986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:24.324986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.358124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1390/2809] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:24.370976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:24.376577 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:24.382587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f847cc-143b-4fef-96a2-0ae317e25400 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:24.387452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:24.388223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.388575 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.388862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] PATH_INFO: `/v3/projects/bef0cfe9c3404a128be955d4fac0eb5a/users/50259072d17a43f083f109cc8173dc02/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.407126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef0cfe9c3404a128be955d4fac0eb5a, user_id=50259072d17a43f083f109cc8173dc02, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:24.410779 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.507406 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1420/2810] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:24 2022] PUT /identity/v3/projects/bef0cfe9c3404a128be955d4fac0eb5a/users/50259072d17a43f083f109cc8173dc02/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:24.511653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:24.523181 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d41d4-4f0e-487a-a901-6fd91e30f4fa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:24.588009 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:24.596001 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff75dc21-f3d5-4f09-a1ec-85a1c4894333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:24.602280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75dc21-f3d5-4f09-a1ec-85a1c4894333 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.602280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75dc21-f3d5-4f09-a1ec-85a1c4894333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.602280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff75dc21-f3d5-4f09-a1ec-85a1c4894333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.602280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75dc21-f3d5-4f09-a1ec-85a1c4894333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:24.602280 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff75dc21-f3d5-4f09-a1ec-85a1c4894333 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.610682 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1391/2811] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:24.615395 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:24.629176 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16b26bc-da1f-4d2d-bf58-92a5ad21b28e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:24.634740 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:24.635509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.635800 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.636084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] PATH_INFO: `/v3/projects/bef0cfe9c3404a128be955d4fac0eb5a/users/50259072d17a43f083f109cc8173dc02/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.638235 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] RBAC: Authorizing `identity:create_grant(project_id=bef0cfe9c3404a128be955d4fac0eb5a, user_id=50259072d17a43f083f109cc8173dc02, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:24.640910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.654982 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1421/2812] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:24 2022] PUT /identity/v3/projects/bef0cfe9c3404a128be955d4fac0eb5a/users/50259072d17a43f083f109cc8173dc02/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:24.671649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.672085 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.672508 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.717069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] MFA Rules not processed for user `50259072d17a43f083f109cc8173dc02`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:24.717069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:24.797915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1392/2813] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:24.807242 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:24.812175 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:24.881110 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d9812-4902-4077-bf1b-9d91ab4b97be admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:24.885405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e7afc-2eda-4c18-ab24-324de396d7ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:24.886076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e7afc-2eda-4c18-ab24-324de396d7ae service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:24.886439 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e7afc-2eda-4c18-ab24-324de396d7ae service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:24.886846 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e7afc-2eda-4c18-ab24-324de396d7ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:24.898265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e7afc-2eda-4c18-ab24-324de396d7ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:24.899718 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e7afc-2eda-4c18-ab24-324de396d7ae service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:24.910777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1422/2814] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:24.941646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:25.014994 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078138d2-aa13-465d-b28b-664a08a461c8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:25.021742 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:25.025231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:25.025231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:25.025231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] PATH_INFO: `/v3/users/1ff33eb661884f19831f4cacfe54047b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:25.025231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff33eb661884f19831f4cacfe54047b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:25.026958 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:25.263772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff33eb661884f19831f4cacfe54047b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:25.270908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff33eb661884f19831f4cacfe54047b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:25.335038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff33eb661884f19831f4cacfe54047b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:25.335662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff33eb661884f19831f4cacfe54047b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:25.350047 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff33eb661884f19831f4cacfe54047b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:25.351761 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:25.356911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1393/2815] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:24 2022] DELETE /identity/v3/users/1ff33eb661884f19831f4cacfe54047b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:25.361430 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:25.361922 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:25.362219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:25.382211 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:25.383436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:25.448940 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1423/2816] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:25.459599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:25.461041 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:25.528024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec0e54-5279-401a-856c-c7b20dc46e3e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:25.528024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:25.528024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:25.528024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:25.528673 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:25.535052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:25.536812 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:25.547240 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1394/2817] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:26.033814 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:26.035587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:26.051174 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfacc26b-b85e-40a6-99b3-c5c2e8a609cd None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:26.056797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:26.057444 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:26.057847 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:26.058338 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] PATH_INFO: `/v3/projects/0430d4f343034016b189217c9d615ad0` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:26.059452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0430d4f343034016b189217c9d615ad0)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:26.061394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:26.188777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0430d4f343034016b189217c9d615ad0', 'request_id': 'req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.272573 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0430d4f343034016b189217c9d615ad0', 'request_id': 'req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.279667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0430d4f343034016b189217c9d615ad0', 'request_id': 'req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.281710 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:26.292030 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] The token cache is being invalidate because project 0430d4f343034016b189217c9d615ad0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:26.292327 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.292987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.294757 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1424/2818] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:26 2022] DELETE /identity/v3/projects/0430d4f343034016b189217c9d615ad0 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:26.299295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:26.310058 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87799-953d-471a-972c-5c69acb70388 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:26.373385 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:26.374110 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:26.374398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:26.374689 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] PATH_INFO: `/v3/users/50259072d17a43f083f109cc8173dc02` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:26.375838 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50259072d17a43f083f109cc8173dc02)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:26.377798 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:26.698387 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50259072d17a43f083f109cc8173dc02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.705317 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50259072d17a43f083f109cc8173dc02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.780851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50259072d17a43f083f109cc8173dc02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.781488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50259072d17a43f083f109cc8173dc02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.803801 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8b551813-4f0e-4c85-9ca8-c67bd880f11c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50259072d17a43f083f109cc8173dc02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:26.809885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1395/2819] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:26 2022] DELETE /identity/v3/users/50259072d17a43f083f109cc8173dc02 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:27.428591 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.430173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:27.444222 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0f4a0-1cf2-40db-8fe3-c2f1ff4ccadc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.525029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.525825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.526038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.526314 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] PATH_INFO: `/v3/projects/bef0cfe9c3404a128be955d4fac0eb5a` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.527811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bef0cfe9c3404a128be955d4fac0eb5a)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.530299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.654126 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef0cfe9c3404a128be955d4fac0eb5a', 'request_id': 'req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.774281 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef0cfe9c3404a128be955d4fac0eb5a', 'request_id': 'req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.785150 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef0cfe9c3404a128be955d4fac0eb5a', 'request_id': 'req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.801502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] The token cache is being invalidate because project bef0cfe9c3404a128be955d4fac0eb5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:27.801833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.802676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-eafc7d7e-5e29-41cb-8789-c5ad2a4c5f9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:27.804318 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1425/2820] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:27 2022] DELETE /identity/v3/projects/bef0cfe9c3404a128be955d4fac0eb5a => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:27.810122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.820533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.820533 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.831660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:27.832908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:27.846774 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:27.895659 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1396/2821] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:27.903975 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.904315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.904547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.926628 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:27.927859 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:27.935827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1426/2822] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:27.944988 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.954875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f43063-5feb-4d62-a213-162139fcab43 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:27.959918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:27.960523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:27.960805 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:27.961080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:27.961755 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:27.962112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:27.963690 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:27.976056 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1397/2823] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:27.981061 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:27.994751 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6783c1bd-b4ba-4818-8aef-90f88bf198dc None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.001059 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.001678 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.002283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.002405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.002989 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.005851 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.153998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1427/2824] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:27 2022] POST /identity/v3/projects => generated 393 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:28.159892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.172470 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78e44-fbf3-40c2-8294-809eea3d4585 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.232176 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.232791 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.233089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.233366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.234044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.235997 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.365415 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1398/2825] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:28.369653 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.380463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1764bdf-411c-43c1-8b22-160cb8078f79 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.385004 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.385662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.386021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.386304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.387043 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.388771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.402584 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1428/2826] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:28.406764 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.421911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f883a-f3f7-47ef-aec9-656cba584d28 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.423396 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764924e-34af-48c4-89f5-1af671417cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.424035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764924e-34af-48c4-89f5-1af671417cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.424382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764924e-34af-48c4-89f5-1af671417cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.424687 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764924e-34af-48c4-89f5-1af671417cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.426057 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764924e-34af-48c4-89f5-1af671417cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.427685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764924e-34af-48c4-89f5-1af671417cd4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.438446 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1399/2827] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:28.442886 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.453714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520924-a342-414b-ba35-46cf48e5430b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.458852 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.459601 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.459890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.460169 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] PATH_INFO: `/v3/projects/cd14cbd387ab42f2917868c5b5a8be75/users/f5a4d2f5ffc447329451c0c4807fcbb2/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.477079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd14cbd387ab42f2917868c5b5a8be75, user_id=f5a4d2f5ffc447329451c0c4807fcbb2, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.479855 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.572208 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1429/2828] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:28 2022] PUT /identity/v3/projects/cd14cbd387ab42f2917868c5b5a8be75/users/f5a4d2f5ffc447329451c0c4807fcbb2/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:28.583253 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.583646 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.584099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.614215 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] MFA Rules not processed for user `f5a4d2f5ffc447329451c0c4807fcbb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:28.615606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:28.690800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1400/2829] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:28.695684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.717266 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:28.762976 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6801ac-6d9c-47b0-8d56-1f28436bc8ba admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.767918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.771423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.771423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.771423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.771423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.773156 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.866671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.868067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:28.882889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d214cb-fca7-4d46-946a-fd7a6557469d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.889658 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.890825 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.891290 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.891749 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] PATH_INFO: `/v3/users/4905d31c7ab9450e99b6e2f69dd42431` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.893633 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4905d31c7ab9450e99b6e2f69dd42431)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.899383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1430/2830] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/projects => generated 393 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:28.899603 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:28.900796 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:28.917773 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:28.977303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d9002-c477-4ae7-8517-c5c0cf7a5923 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:28.982203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:28.982904 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:28.983097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:28.983376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:28.984021 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:28.986076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.136232 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1431/2831] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:28 2022] POST /identity/v3/users => generated 303 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:29.140321 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.151244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4cc969-6437-405a-b230-7b3458c50b3d admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.155903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.156504 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.156782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.157055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.157811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.159310 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.160935 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4905d31c7ab9450e99b6e2f69dd42431'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:29.170312 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4905d31c7ab9450e99b6e2f69dd42431'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:29.173987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1432/2832] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:29.178516 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.188957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d642d-ccc4-4197-bd7c-99abf66ed21e admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.196781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.196781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.196781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.196781 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] PATH_INFO: `/v3/projects/b2434e64cd164be6a1efb346e840315e/users/5f7fd3263b414049b1a27b8153e0f0c3/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.212464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2434e64cd164be6a1efb346e840315e, user_id=5f7fd3263b414049b1a27b8153e0f0c3, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.215586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.255248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4905d31c7ab9450e99b6e2f69dd42431'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:29.255980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4905d31c7ab9450e99b6e2f69dd42431'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:29.269691 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bf55c560-ad3d-487b-88dc-9a8783b3a4c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4905d31c7ab9450e99b6e2f69dd42431'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:29.276376 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1401/2833] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:28 2022] DELETE /identity/v3/users/4905d31c7ab9450e99b6e2f69dd42431 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:29.280947 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.281241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.281522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.300926 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1433/2834] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:29 2022] PUT /identity/v3/projects/b2434e64cd164be6a1efb346e840315e/users/5f7fd3263b414049b1a27b8153e0f0c3/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:29.304816 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:29.304998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.306287 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.306671 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.370988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:29.373915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1402/2835] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:29.375677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59450135-3482-421b-91b9-b2b794dbe652 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.382182 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.382818 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.383196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.383558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.384209 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.384620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.385914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.386174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.397165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1434/2836] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:29.401308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.412588 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e8f6e-58d5-40de-85c8-a08e98ce7a22 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.417908 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84452db6-dfe3-4fbd-9208-2ff62bed2060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.418607 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84452db6-dfe3-4fbd-9208-2ff62bed2060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.418884 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84452db6-dfe3-4fbd-9208-2ff62bed2060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.419157 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84452db6-dfe3-4fbd-9208-2ff62bed2060 admin admin] PATH_INFO: `/v3/projects/b2434e64cd164be6a1efb346e840315e/users/5f7fd3263b414049b1a27b8153e0f0c3/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.421075 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84452db6-dfe3-4fbd-9208-2ff62bed2060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2434e64cd164be6a1efb346e840315e, user_id=5f7fd3263b414049b1a27b8153e0f0c3, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.423766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84452db6-dfe3-4fbd-9208-2ff62bed2060 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.434486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1435/2837] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:29 2022] PUT /identity/v3/projects/b2434e64cd164be6a1efb346e840315e/users/5f7fd3263b414049b1a27b8153e0f0c3/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:29.460912 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e254b5-a798-4cd3-938d-04ee3d28d40d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.504076 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.504909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.504909 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.521198 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:29.531448 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.535418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.535418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.535418 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.535821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] MFA Rules not processed for user `5f7fd3263b414049b1a27b8153e0f0c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:29.537474 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.604024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.604024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.616369 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1436/2838] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:29.617890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1403/2839] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:29.627191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.630079 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.700452 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8df5f9-f5d8-49f9-a95d-d941e565ccaa None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.704244 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.704858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.705213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.705494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.717372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.718916 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.729548 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1437/2840] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:29.779589 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:29.781038 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.793419 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e345052-c8f4-4172-a84c-f086bee54a13 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:29.797059 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:29.797663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:29.797980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:29.798259 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:29.800052 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:29.863241 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:29.864810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aeb258-bf4d-4a63-86e8-a2cb1ace2584 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:29.875398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1404/2841] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:29 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:30.137219 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:30.138745 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:30.153749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae13750-86de-465b-9457-a4b4ba3ed91f service cinder] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:30.158804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:30.159639 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.159994 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.160497 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] PATH_INFO: `/v3/projects/df5d9d59d2a246298cb28e971b9b5eca` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.162093 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5d9d59d2a246298cb28e971b9b5eca)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:30.164195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:30.272203 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5d9d59d2a246298cb28e971b9b5eca', 'request_id': 'req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:30.357662 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5d9d59d2a246298cb28e971b9b5eca', 'request_id': 'req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:30.365998 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5d9d59d2a246298cb28e971b9b5eca', 'request_id': 'req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:30.368585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:30.379185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] The token cache is being invalidate because project df5d9d59d2a246298cb28e971b9b5eca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:30.379496 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:30.380412 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b136edc9-8bbb-4f6e-99dc-0c2d864c7b59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:30.381872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1438/2842] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:30 2022] DELETE /identity/v3/projects/df5d9d59d2a246298cb28e971b9b5eca => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:30.390478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.390478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.390478 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.410430 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:30.412049 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:30.478643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1405/2843] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:30.544785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.545360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.545360 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.567551 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:30.568850 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:30.577252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1439/2844] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:30.582307 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:30.592704 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87cd8a0-c8d3-4b96-add6-622ade8c20dd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:30.597516 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:30.598089 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.598374 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.598647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.599309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:30.599593 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:30.601166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:30.613239 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1406/2845] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 01:19:30.617891 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:30.627426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb4be8d-f313-4f16-9210-75b29b013e4d None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:30.632944 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:30.633495 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.633951 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.634228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.634862 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:30.636806 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:30.742788 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1440/2846] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:30 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:30.747265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:30.758717 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9af7f-4cee-4790-98db-477373862ba9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:30.778080 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:30.820107 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:30.820796 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.821103 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.821382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.822070 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:30.824154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:30.930484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1407/2847] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:30 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:30.934631 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:30.945599 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fe603a-a9db-4836-a1dc-b20206300d8b admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:30.951033 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:30.951646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.951957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.952234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.952887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:30.954547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:30.964597 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1441/2848] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:30.968793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:30.979538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd53b0-7623-4143-8aa9-0956a15c65c4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:30.984599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6d7d33-084e-472e-aefe-212107f2ce3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:30.985205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7d33-084e-472e-aefe-212107f2ce3b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:30.985933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7d33-084e-472e-aefe-212107f2ce3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:30.985933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6d7d33-084e-472e-aefe-212107f2ce3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:30.986654 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7d33-084e-472e-aefe-212107f2ce3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:30.988848 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6d7d33-084e-472e-aefe-212107f2ce3b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:30.999957 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1408/2849] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:31.004997 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:31.017568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65282979-41e9-4b44-9c82-2ad2c6c970c5 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:31.022381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:31.023028 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:31.023308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:31.023582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] PATH_INFO: `/v3/projects/3068817b4ca34beca8bd4da5409e58ef/users/b5b9eb7e61704da582a326e48e3a5170/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:31.038914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=3068817b4ca34beca8bd4da5409e58ef, user_id=b5b9eb7e61704da582a326e48e3a5170, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:31.041568 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:31.101750 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1442/2850] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:31 2022] PUT /identity/v3/projects/3068817b4ca34beca8bd4da5409e58ef/users/b5b9eb7e61704da582a326e48e3a5170/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:31.107322 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:31.107620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:31.107903 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:31.128106 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:31.130013 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:31.191033 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1409/2851] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:31.200677 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:31.202586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:31.260776 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effd04b-4f1a-4100-9abd-bcc55d6ee9ce admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:31.264532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a4ceb-bcab-498e-92de-4e7073a4a677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:31.265194 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4ceb-bcab-498e-92de-4e7073a4a677 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:31.265478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4ceb-bcab-498e-92de-4e7073a4a677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:31.265775 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a4ceb-bcab-498e-92de-4e7073a4a677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:31.275274 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a4ceb-bcab-498e-92de-4e7073a4a677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:31.276758 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3a4ceb-bcab-498e-92de-4e7073a4a677 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:31.287201 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1443/2852] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:35.180451 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:35.182154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:35.195114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b22d4-79e9-4640-88cb-bae77e63f0f6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:35.200182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:35.200831 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:35.201112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:35.201389 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] PATH_INFO: `/v3/users/f5a4d2f5ffc447329451c0c4807fcbb2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:35.202597 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a4d2f5ffc447329451c0c4807fcbb2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:35.204598 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:35.417558 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a4d2f5ffc447329451c0c4807fcbb2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:35.427005 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a4d2f5ffc447329451c0c4807fcbb2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:35.500703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a4d2f5ffc447329451c0c4807fcbb2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:35.501459 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a4d2f5ffc447329451c0c4807fcbb2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:35.517390 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a4d2f5ffc447329451c0c4807fcbb2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:35.527468 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1410/2853] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:35 2022] DELETE /identity/v3/users/f5a4d2f5ffc447329451c0c4807fcbb2 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:35.532478 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:35.533000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:35.534386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:35.561887 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:35.563224 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:35.627424 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1444/2854] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:35.638737 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:35.640225 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:35.654965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:35.720671 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119ee441-3571-4a48-837c-0a6ae867d64f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:35.726358 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:35.726991 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:35.727327 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:35.727840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:35.741270 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:35.743456 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:35.756846 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1411/2855] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:36.420373 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:36.422879 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:36.445387 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333aad65-0ee6-4553-8940-89391eba4917 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:36.450629 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:36.451282 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:36.451713 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:36.451936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] PATH_INFO: `/v3/projects/cd14cbd387ab42f2917868c5b5a8be75` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:36.453103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd14cbd387ab42f2917868c5b5a8be75)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:36.455210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:36.525550 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:36.551830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd14cbd387ab42f2917868c5b5a8be75', 'request_id': 'req-c3a47255-e514-4d12-bd01-915f226d8b16'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.643332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd14cbd387ab42f2917868c5b5a8be75', 'request_id': 'req-c3a47255-e514-4d12-bd01-915f226d8b16'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.656431 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd14cbd387ab42f2917868c5b5a8be75', 'request_id': 'req-c3a47255-e514-4d12-bd01-915f226d8b16'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.671087 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] The token cache is being invalidate because project cd14cbd387ab42f2917868c5b5a8be75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81621) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:36.677618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.677618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-c3a47255-e514-4d12-bd01-915f226d8b16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:36.677618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1445/2856] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:36 2022] DELETE /identity/v3/projects/cd14cbd387ab42f2917868c5b5a8be75 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:36.678990 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:36.692866 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf829fb-e432-46af-a48a-395e062799b0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:36.808771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:36.809476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:36.811295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:36.811295 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] PATH_INFO: `/v3/users/5f7fd3263b414049b1a27b8153e0f0c3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:36.811930 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f7fd3263b414049b1a27b8153e0f0c3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:36.813968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.092339 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f7fd3263b414049b1a27b8153e0f0c3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.104585 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f7fd3263b414049b1a27b8153e0f0c3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.181097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f7fd3263b414049b1a27b8153e0f0c3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.182084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f7fd3263b414049b1a27b8153e0f0c3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.219024 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f7fd3263b414049b1a27b8153e0f0c3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.227970 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1412/2857] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:36 2022] DELETE /identity/v3/users/5f7fd3263b414049b1a27b8153e0f0c3 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:37.298735 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.299024 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.299317 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.343587 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] MFA Rules not processed for user `b5b9eb7e61704da582a326e48e3a5170`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:37.346102 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:37.430848 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1446/2858] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:37 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:37.437959 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.442256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:37.497618 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:37.532325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b643fa7-4187-4e79-8bf7-803b865750a4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.538397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.539040 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.539328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.539621 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.540327 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.542565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.663205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1413/2859] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:37 2022] POST /identity/v3/projects => generated 375 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:37.667401 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.679018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8100155-4083-48dc-952f-476e1b22ca24 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.739225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.739966 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.740247 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.741038 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.741766 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.746986 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.790578 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.793075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:37.816370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1a600-2c24-44ec-98d3-9dd364c0779e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.822362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.823837 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.824122 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.824400 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] PATH_INFO: `/v3/projects/b2434e64cd164be6a1efb346e840315e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.825637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2434e64cd164be6a1efb346e840315e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.827718 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.895308 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1447/2860] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:37 2022] POST /identity/v3/users => generated 294 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:37.899513 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.909355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfaef81-bc8f-4bb1-8643-86c6ee932769 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.915428 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.916029 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.916330 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.916589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.917228 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.918927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:37.929910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1448/2861] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:37.934048 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:37.944425 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8b3d3-5a4f-4989-9f64-6dd2181dfac1 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:37.949486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:37.950225 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:37.950580 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:37.950778 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] PATH_INFO: `/v3/projects/561828cfc1c64d8b89e7e13f39cd6911/users/73d80c58ebfe4e589d3887353b26faf3/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:37.961999 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2434e64cd164be6a1efb346e840315e', 'request_id': 'req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.970984 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2434e64cd164be6a1efb346e840315e', 'request_id': 'req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.978437 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=561828cfc1c64d8b89e7e13f39cd6911, user_id=73d80c58ebfe4e589d3887353b26faf3, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:37.979892 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2434e64cd164be6a1efb346e840315e', 'request_id': 'req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:37.982777 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.060077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] The token cache is being invalidate because project b2434e64cd164be6a1efb346e840315e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:38.060404 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.061055 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-fe6b90ed-06e1-4fb6-a99a-f8a45c799aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:38.062800 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1414/2862] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:37 2022] DELETE /identity/v3/projects/b2434e64cd164be6a1efb346e840315e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:38.067251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:38.068902 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a05ff-dd42-4166-b2ce-1a037aae3d2b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.069391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a05ff-dd42-4166-b2ce-1a037aae3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.069936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0a05ff-dd42-4166-b2ce-1a037aae3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.080500 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1449/2863] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:37 2022] PUT /identity/v3/projects/561828cfc1c64d8b89e7e13f39cd6911/users/73d80c58ebfe4e589d3887353b26faf3/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:38.083368 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.085610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:38.094975 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ed0a05ff-dd42-4166-b2ce-1a037aae3d2b None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:38.096370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ed0a05ff-dd42-4166-b2ce-1a037aae3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:38.160130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c430d9-bbfe-41b0-9723-c144ed13a9be admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.166658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.167159 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1415/2864] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:38.167442 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.167722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.168003 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.168655 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.170337 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.174694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.174982 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.175261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.184939 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1450/2865] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 01:19:38.186047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.196811 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba5656-1fbe-4827-8ce1-68d138bd93ad admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.198410 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:38.199636 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:38.202116 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.203074 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.203381 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.203723 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] PATH_INFO: `/v3/projects/561828cfc1c64d8b89e7e13f39cd6911/users/73d80c58ebfe4e589d3887353b26faf3/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.205931 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561828cfc1c64d8b89e7e13f39cd6911, user_id=73d80c58ebfe4e589d3887353b26faf3, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.212173 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1416/2866] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:38.212384 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.213681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.222032 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1451/2867] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:38 2022] PUT /identity/v3/projects/561828cfc1c64d8b89e7e13f39cd6911/users/73d80c58ebfe4e589d3887353b26faf3/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:38.223244 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c56507a-c9db-48f6-8989-9155ed9b9323 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.261868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:38.294436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.294436 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.294856 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.299995 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.304783 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:38.305171 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.307333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.325341 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1417/2868] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:38.326315 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.339936 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff416706-e4fc-436a-9e3e-09be5939a4e2 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.354730 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.355618 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.356114 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.356626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.358421 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.362014 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.465771 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1452/2869] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:38 2022] POST /identity/v3/projects => generated 387 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 01:19:38.467151 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.478174 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8c9db-76b2-4b66-bc4a-07418005d4be admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.554235 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.555045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.555502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.555908 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.556642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.559379 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.639232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1418/2870] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:38 2022] POST /identity/v3/users => generated 294 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:38.643285 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.656517 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa77cdc-3d30-4b00-bc26-23847f6cf089 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.662000 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.662609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.662889 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.663230 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.663890 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.665586 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.676674 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1453/2871] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:38.680435 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.692896 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9825c76-246d-4a8a-b0ae-78e8524acd1e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.699063 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a0aa90-74be-43d4-addb-975e8cfc4b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.699899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0aa90-74be-43d4-addb-975e8cfc4b81 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.700614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0aa90-74be-43d4-addb-975e8cfc4b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.700614 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a0aa90-74be-43d4-addb-975e8cfc4b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.701168 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0aa90-74be-43d4-addb-975e8cfc4b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.702794 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a0aa90-74be-43d4-addb-975e8cfc4b81 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.718394 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1419/2872] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:38.719334 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:38.732223 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ba7de-52eb-46a1-81db-eb98d5a7dd91 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:38.740303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:38.741304 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.741841 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.742364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] PATH_INFO: `/v3/projects/5f44e6ea3fd5404d974b929b3aee88d1/users/5fed5f78f46648fbb397c32edf4f1608/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.760703 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f44e6ea3fd5404d974b929b3aee88d1, user_id=5fed5f78f46648fbb397c32edf4f1608, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:38.764136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:38.870685 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1454/2873] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:38 2022] PUT /identity/v3/projects/5f44e6ea3fd5404d974b929b3aee88d1/users/5fed5f78f46648fbb397c32edf4f1608/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:38.883918 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:38.884489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:38.884489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:38.915562 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] MFA Rules not processed for user `5fed5f78f46648fbb397c32edf4f1608`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:38.916806 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:38.991120 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1420/2874] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:38.995583 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.078295 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb956e-9599-4747-9d68-2785fb422e2f admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.084885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.085531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.086248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.086248 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.086797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.089180 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.211675 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1455/2875] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:38 2022] POST /identity/v3/projects => generated 389 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:39.215799 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.229899 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c58e1-2e99-4b8e-ad59-1d14eaf2a10e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.274127 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:39.313890 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.314728 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.315088 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.315442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.317545 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.321315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.419797 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1421/2876] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:39 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:39.421246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.432812 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc37809-3bad-4650-a33a-ea3897315e3a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.438165 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.438821 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.439103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.443785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.443785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.443785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.444954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:39.457488 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1456/2877] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:39.458725 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.483968 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f737b0-6382-4641-8055-bc585b8873ae admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.489681 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.490324 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.490641 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.491560 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] PATH_INFO: `/v3/projects/4a36ef3ddc884e24ae3b1a8f9d32559c/users/068eb602bf32409591398e41892ec581/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.507939 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a36ef3ddc884e24ae3b1a8f9d32559c, user_id=068eb602bf32409591398e41892ec581, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.512370 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.627980 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1422/2878] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:39 2022] PUT /identity/v3/projects/4a36ef3ddc884e24ae3b1a8f9d32559c/users/068eb602bf32409591398e41892ec581/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:39.632572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.644137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8780d81d-1e8b-4069-8538-eb7eb7fb87ab admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.704726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474e0ad-b511-439d-9556-fdd68da691a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.705352 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474e0ad-b511-439d-9556-fdd68da691a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.705637 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474e0ad-b511-439d-9556-fdd68da691a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.705953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474e0ad-b511-439d-9556-fdd68da691a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.706596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474e0ad-b511-439d-9556-fdd68da691a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.709185 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8474e0ad-b511-439d-9556-fdd68da691a2 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.720833 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1457/2879] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:39 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:39.724480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.735382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0457ec-6d3d-45cc-ad8b-f152807b86f7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.740845 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.741518 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.741832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.742115 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] PATH_INFO: `/v3/projects/4a36ef3ddc884e24ae3b1a8f9d32559c/users/068eb602bf32409591398e41892ec581/roles/9d7628878120443f8b9950c60d7a8671` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.744196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a36ef3ddc884e24ae3b1a8f9d32559c, user_id=068eb602bf32409591398e41892ec581, role_id=9d7628878120443f8b9950c60d7a8671)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.746915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:39.758785 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1423/2880] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:39 2022] PUT /identity/v3/projects/4a36ef3ddc884e24ae3b1a8f9d32559c/users/068eb602bf32409591398e41892ec581/roles/9d7628878120443f8b9950c60d7a8671 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:39.774957 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.775697 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.776101 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.812785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] MFA Rules not processed for user `068eb602bf32409591398e41892ec581`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:39.814193 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:39.891394 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1458/2881] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:39 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:39.902972 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:39.904712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:39.977642 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e83364-e52a-43c6-bf0a-fda05aeda264 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:39.982476 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8d7b6e-1b3c-43a8-9e7d-306964826322 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:39.983293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d7b6e-1b3c-43a8-9e7d-306964826322 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:39.983669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d7b6e-1b3c-43a8-9e7d-306964826322 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:39.984044 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8d7b6e-1b3c-43a8-9e7d-306964826322 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:39.994684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d7b6e-1b3c-43a8-9e7d-306964826322 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:39.996333 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8d7b6e-1b3c-43a8-9e7d-306964826322 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:40.008437 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1424/2882] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:39 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:40.168741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:40.252789 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17e4a6-c92e-4b4a-804a-6237d6ec7fd9 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:40.259356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:40.260053 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:40.260297 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:40.260578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] PATH_INFO: `/v3/users/5fed5f78f46648fbb397c32edf4f1608` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:40.261882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fed5f78f46648fbb397c32edf4f1608)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:40.264170 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:40.300086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:40.516633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fed5f78f46648fbb397c32edf4f1608'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:40.527995 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fed5f78f46648fbb397c32edf4f1608'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:40.529120 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fed5f78f46648fbb397c32edf4f1608'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:40.531531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fed5f78f46648fbb397c32edf4f1608'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:40.547467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fed5f78f46648fbb397c32edf4f1608'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:40.647950 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1459/2883] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:40 2022] DELETE /identity/v3/users/5fed5f78f46648fbb397c32edf4f1608 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:40.652592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:40.652945 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:40.653669 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:40.676233 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:40.677502 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:40.776606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1425/2884] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:40.789007 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:40.791582 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:40.875691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7707-b682-4e75-8ce3-bfd289231b69 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:40.884534 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:40.885172 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:40.885451 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:40.885749 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:40.906971 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:40.908589 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:40.920695 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1460/2885] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:41.459211 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:41.462638 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:41.477680 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73e097-4a0e-42b0-8b06-d7e9c0321915 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:41.482844 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:41.483500 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:41.483931 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:41.484398 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] PATH_INFO: `/v3/projects/5f44e6ea3fd5404d974b929b3aee88d1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:41.485762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f44e6ea3fd5404d974b929b3aee88d1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:41.487731 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:41.578521 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:41.601248 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f44e6ea3fd5404d974b929b3aee88d1', 'request_id': 'req-5d7c8458-65d2-404d-9b30-59e8f05e8f73'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:41.609787 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f44e6ea3fd5404d974b929b3aee88d1', 'request_id': 'req-5d7c8458-65d2-404d-9b30-59e8f05e8f73'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:41.620693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f44e6ea3fd5404d974b929b3aee88d1', 'request_id': 'req-5d7c8458-65d2-404d-9b30-59e8f05e8f73'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:41.723900 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] The token cache is being invalidate because project 5f44e6ea3fd5404d974b929b3aee88d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:41.724300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:41.725133 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:41.729933 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1426/2886] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:41 2022] DELETE /identity/v3/projects/5f44e6ea3fd5404d974b929b3aee88d1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:41.730987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:41.746288 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a5206-6eb7-48bc-82a5-1f5ffbcb4178 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:41.835249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:41.836066 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:41.836354 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:41.838785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] PATH_INFO: `/v3/users/068eb602bf32409591398e41892ec581` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:41.838785 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068eb602bf32409591398e41892ec581)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:41.840283 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:42.103552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:42.107196 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068eb602bf32409591398e41892ec581'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:42.115341 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068eb602bf32409591398e41892ec581'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:42.117188 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068eb602bf32409591398e41892ec581'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:42.120706 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068eb602bf32409591398e41892ec581'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:42.136042 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-36ec1630-d15d-4468-9148-2b9dee669c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068eb602bf32409591398e41892ec581'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:42.255611 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1461/2887] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:41 2022] DELETE /identity/v3/users/068eb602bf32409591398e41892ec581 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:19:43.441606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:43.441606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:43.473515 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7c8458-65d2-404d-9b30-59e8f05e8f73 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:43.592196 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:43.593541 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:43.599356 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:43.599965 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] PATH_INFO: `/v3/projects/4a36ef3ddc884e24ae3b1a8f9d32559c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:43.602111 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a36ef3ddc884e24ae3b1a8f9d32559c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:43.605917 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:43.789362 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a36ef3ddc884e24ae3b1a8f9d32559c', 'request_id': 'req-920810f0-8e10-4e7d-93a9-429a1f2e54a5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:43.800039 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a36ef3ddc884e24ae3b1a8f9d32559c', 'request_id': 'req-920810f0-8e10-4e7d-93a9-429a1f2e54a5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:43.810325 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a36ef3ddc884e24ae3b1a8f9d32559c', 'request_id': 'req-920810f0-8e10-4e7d-93a9-429a1f2e54a5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:43.920360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] The token cache is being invalidate because project 4a36ef3ddc884e24ae3b1a8f9d32559c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:43.920662 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:43.922426 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:43.924861 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1427/2888] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:43 2022] DELETE /identity/v3/projects/4a36ef3ddc884e24ae3b1a8f9d32559c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:44.967309 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:44.967827 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:44.968054 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.000501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] MFA Rules not processed for user `73d80c58ebfe4e589d3887353b26faf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:45.001722 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.078191 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1462/2889] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:45.095792 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:45.102309 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.154286 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:45.175276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920810f0-8e10-4e7d-93a9-429a1f2e54a5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:45.181021 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:45.181649 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:45.181962 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:45.182238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:45.184150 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:45.246526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:45.248126 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe17b9-108e-4044-90cb-9946e7fbc72f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:45.257952 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1428/2890] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:55.074612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:DELETE=4,keystone:SELECT=47,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:55.577576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:55.579204 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:55.648109 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ffc30e-afbb-4f69-9f10-543327e656e7 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:55.652726 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:55.653363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:55.653643 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:55.653954 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] PATH_INFO: `/v3/users/051c8050236a4882b5c1187a8f403981` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:55.655117 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051c8050236a4882b5c1187a8f403981)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:55.657055 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:55.871131 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051c8050236a4882b5c1187a8f403981'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:55.878271 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051c8050236a4882b5c1187a8f403981'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:55.878965 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051c8050236a4882b5c1187a8f403981'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:55.879782 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051c8050236a4882b5c1187a8f403981'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:55.891918 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051c8050236a4882b5c1187a8f403981'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:56.003031 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1463/2891] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:19:55 2022] DELETE /identity/v3/users/051c8050236a4882b5c1187a8f403981 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:56.007482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:56.008025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:56.008025 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:56.027851 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:56.029053 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:56.106703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1429/2892] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:56.116902 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:56.118560 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:56.181134 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919f0bd-51c3-4501-9731-ab0a90dbb710 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:56.184897 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aad53a-fecd-419e-8d0f-de210fb558aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:56.185512 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aad53a-fecd-419e-8d0f-de210fb558aa service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:56.185829 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aad53a-fecd-419e-8d0f-de210fb558aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:56.186108 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aad53a-fecd-419e-8d0f-de210fb558aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:56.195972 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aad53a-fecd-419e-8d0f-de210fb558aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:56.197427 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aad53a-fecd-419e-8d0f-de210fb558aa service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:56.207446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1464/2893] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:19:56.639202 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:56.640667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:56.655357 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd705cb7-d57b-4e6e-bc5a-da15812bbaa3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:56.661397 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:56.662164 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:56.662447 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:56.662741 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] PATH_INFO: `/v3/projects/6d66050655494b9880890460ef7ee8b5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:56.665002 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d66050655494b9880890460ef7ee8b5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:56.667051 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:56.786373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d66050655494b9880890460ef7ee8b5', 'request_id': 'req-ad4e3967-d348-483a-94a7-0523f75229ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:56.792702 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d66050655494b9880890460ef7ee8b5', 'request_id': 'req-ad4e3967-d348-483a-94a7-0523f75229ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:56.798772 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d66050655494b9880890460ef7ee8b5', 'request_id': 'req-ad4e3967-d348-483a-94a7-0523f75229ce'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:56.865145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] The token cache is being invalidate because project 6d66050655494b9880890460ef7ee8b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:19:56.865472 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:56.866146 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ad4e3967-d348-483a-94a7-0523f75229ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:19:56.867696 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1430/2894] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:19:56 2022] DELETE /identity/v3/projects/6d66050655494b9880890460ef7ee8b5 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:56.872747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:56.873039 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:56.873323 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:56.892870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:56.894092 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:56.912419 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:56.957824 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1465/2895] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:56.966495 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:56.966786 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:56.967069 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:56.982601 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:56.988455 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:56.991886 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:56.998261 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1431/2896] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:57.002658 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.014510 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce688b-1985-419d-9c36-000ad14979ef None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.019289 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.023432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.023432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.023432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.023432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81621) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 01:19:57.023432 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.024094 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.036843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1466/2897] 10.222.0.52 () {58 vars in 1153 bytes} [Tue Aug 30 01:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 01:19:57.041936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.052042 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346abf7-2afa-4e5b-b2ec-54bc023b4ae1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.057368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.058117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.058401 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.058676 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.059382 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.061443 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.141656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1432/2898] 10.222.0.52 () {60 vars in 1153 bytes} [Tue Aug 30 01:19:57 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:57.145988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.157553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95651b44-c01d-4770-99d7-56a14dad2e78 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.216606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.217215 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.217498 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.217843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] PATH_INFO: `/v3/users` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.218446 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.220372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.295572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1467/2899] 10.222.0.52 () {60 vars in 1144 bytes} [Tue Aug 30 01:19:57 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 01:19:57.299647 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.310572 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4fb94-bc79-446b-ad25-32da91b2210c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.315599 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.316208 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.316496 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.316777 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.317536 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.319262 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.329606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1433/2900] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:57.334190 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.345078 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805cf4b-f3c1-4d25-a3f1-6aa5436cb0df admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.351249 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac2c1ef-1e97-4f5f-9311-0c5e1be33824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.351867 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c1ef-1e97-4f5f-9311-0c5e1be33824 admin admin] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.352144 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c1ef-1e97-4f5f-9311-0c5e1be33824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.352420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac2c1ef-1e97-4f5f-9311-0c5e1be33824 admin admin] PATH_INFO: `/v3/roles` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.353064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c1ef-1e97-4f5f-9311-0c5e1be33824 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.354717 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac2c1ef-1e97-4f5f-9311-0c5e1be33824 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.366258 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1468/2901] 10.222.0.52 () {58 vars in 1122 bytes} [Tue Aug 30 01:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 01:19:57.370637 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.380770 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0f53f-606b-4129-8fda-4ad7364b10cf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.386314 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.386981 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.387265 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.387547 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] PATH_INFO: `/v3/projects/6bb93b63909346cbab03513b7707f7b8/users/998bb52979154fdc824ec07589df82c6/roles/f8e7032fac524859a7ef27220eb9a8fd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.404548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb93b63909346cbab03513b7707f7b8, user_id=998bb52979154fdc824ec07589df82c6, role_id=f8e7032fac524859a7ef27220eb9a8fd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.407222 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.500299 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1434/2902] 10.222.0.52 () {60 vars in 1483 bytes} [Tue Aug 30 01:19:57 2022] PUT /identity/v3/projects/6bb93b63909346cbab03513b7707f7b8/users/998bb52979154fdc824ec07589df82c6/roles/f8e7032fac524859a7ef27220eb9a8fd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:19:57.505610 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448091-886c-4df6-87dd-81172979419e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.506115 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448091-886c-4df6-87dd-81172979419e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.507073 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d448091-886c-4df6-87dd-81172979419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.526594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-9d448091-886c-4df6-87dd-81172979419e None None] MFA Rules not processed for user `efbdbf875e6a427e9e8f14457c4de416`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:57.527872 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-9d448091-886c-4df6-87dd-81172979419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:57.596463 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1469/2903] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 01:19:57.606483 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:57.608624 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:57.671941 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a1cd25-361a-4866-8c13-c769127d30f6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:57.675620 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:57.676231 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:57.676540 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:57.676819 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:57.686685 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:57.688117 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:57.697956 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1435/2904] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 01:19:59.027539 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d17bb-1187-49d5-a78e-4a2213a8322d None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:59.028413 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d17bb-1187-49d5-a78e-4a2213a8322d None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:59.028594 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2d17bb-1187-49d5-a78e-4a2213a8322d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:59.062448 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-3a2d17bb-1187-49d5-a78e-4a2213a8322d None None] MFA Rules not processed for user `998bb52979154fdc824ec07589df82c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:19:59.063598 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-3a2d17bb-1187-49d5-a78e-4a2213a8322d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:59.081574 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:19:59.136290 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1470/2905] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:19:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:19:59.147651 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:19:59.150075 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:19:59.216457 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2dcb9-1bb5-4832-8706-acd243791612 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:19:59.220229 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d427cf-4fe6-4d2a-b157-9d8d6a6426f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:19:59.220949 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d427cf-4fe6-4d2a-b157-9d8d6a6426f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:19:59.221238 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d427cf-4fe6-4d2a-b157-9d8d6a6426f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:19:59.221513 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d427cf-4fe6-4d2a-b157-9d8d6a6426f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:19:59.233061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d427cf-4fe6-4d2a-b157-9d8d6a6426f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:19:59.234554 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d427cf-4fe6-4d2a-b157-9d8d6a6426f8 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:19:59.244523 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1436/2906] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:19:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:20:01.152420 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:01.152959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:01.152959 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:01.172840 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] MFA Rules not processed for user `998bb52979154fdc824ec07589df82c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:01.174137 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:01.181830 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1471/2907] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:20:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:06.879889 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:06.880408 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:06.880538 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:06.901537 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] MFA Rules not processed for user `b5b9eb7e61704da582a326e48e3a5170`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:20:06.902804 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:06.956644 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:06.968684 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1437/2908] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:20:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:20:11.167084 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=22 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:17.007703 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:27.059136 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:27.069119 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b552996-9f28-478f-bd89-b3bc468879bb None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:27.073213 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:27.074047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:27.074335 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:27.074612 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:27.076353 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:27.143254 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:27.144626 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:27.155714 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1472/2909] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:20:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:20:37.150364 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=30 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:20:57.598351 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:20:57.599841 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:57.613643 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539ce182-b6b5-43ea-9090-a3f2a092e029 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:20:57.617653 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:20:57.618360 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:20:57.618723 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:20:57.619077 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:20:57.621511 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:20:57.695752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:20:57.697197 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b969f-a001-45cd-9fc0-601f3e20a8ef service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:20:57.709571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1438/2910] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:20:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:00.925826 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:00.927332 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:00.938576 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8161c76-67bf-43bb-8107-15cf6e433c70 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:00.942386 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:00.942981 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:00.943259 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:00.943721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:00.945467 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:00.959941 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:00.961382 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:00.973105 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1473/2911] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:07.705067 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=34 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:10.968246 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:14.835154 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:14.835619 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:14.835752 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:14.855986 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] MFA Rules not processed for user `998bb52979154fdc824ec07589df82c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:21:14.857655 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:14.870885 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1439/2912] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:21:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:21:18.875265 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:18.884927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b81f9-7ec8-4e5d-9fdb-125defd55677 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:18.889623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:18.890316 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:18.890602 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:18.890882 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:18.892704 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:18.904238 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:18.905691 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:18.916097 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1474/2913] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:24.849567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:28.911252 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:33.165166 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:33.175670 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d1a96-f77d-48be-ab03-cc1544daa527 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:33.180266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:33.180266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:33.180266 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:33.180679 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:33.182489 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:33.192613 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:33.194097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:33.203220 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1440/2914] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:33.283688 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:33.292769 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39d9c8-3876-4d02-a2b7-e61044b6ef78 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:33.296393 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:33.296987 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:33.297272 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:33.297549 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:33.299299 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:33.313464 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:33.314937 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:33.327020 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1475/2915] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:21:43.200256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:43.321018 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:21:57.992915 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:21:57.994600 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:58.011872 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e943db9-5eeb-4342-a40e-acb412c8e5f0 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:21:58.017694 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:21:58.018366 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:21:58.018571 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:21:58.018894 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:21:58.021526 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:21:58.036677 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:21:58.038300 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:21:58.051934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1441/2916] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:21:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:08.044771 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:27.901392 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:27.912816 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fc12b3-5084-46f2-af85-9e0a0ae9cd68 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:27.917581 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:27.918251 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:27.918411 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:27.918707 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] PATH_INFO: `/v3/users/998bb52979154fdc824ec07589df82c6` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:27.919870 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=998bb52979154fdc824ec07589df82c6)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:27.922552 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:28.126764 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '998bb52979154fdc824ec07589df82c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.133721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '998bb52979154fdc824ec07589df82c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.134423 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '998bb52979154fdc824ec07589df82c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.135046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '998bb52979154fdc824ec07589df82c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.147186 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-d611e222-92f5-4ed0-bb24-832a1f950845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '998bb52979154fdc824ec07589df82c6'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.234434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1476/2917] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:22:27 2022] DELETE /identity/v3/users/998bb52979154fdc824ec07589df82c6 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:28.547550 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:28.560911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61071b8-3dbd-4e4f-b62e-54b8ba77e47f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:28.618276 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:28.618911 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:28.619187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:28.619462 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] PATH_INFO: `/v3/projects/6bb93b63909346cbab03513b7707f7b8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:28.620575 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb93b63909346cbab03513b7707f7b8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:28.622499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:28.734901 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb93b63909346cbab03513b7707f7b8', 'request_id': 'req-65def593-f7de-4b4a-a98a-5d343199d83a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.741147 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb93b63909346cbab03513b7707f7b8', 'request_id': 'req-65def593-f7de-4b4a-a98a-5d343199d83a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.746919 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb93b63909346cbab03513b7707f7b8', 'request_id': 'req-65def593-f7de-4b4a-a98a-5d343199d83a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.832315 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] The token cache is being invalidate because project 6bb93b63909346cbab03513b7707f7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:22:28.832615 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.833256 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:22:28.834867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1442/2918] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:22:28 2022] DELETE /identity/v3/projects/6bb93b63909346cbab03513b7707f7b8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:22:38.229983 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:38.828976 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:40.192465 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:40.192963 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:40.193063 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:40.213345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] MFA Rules not processed for user `b5b9eb7e61704da582a326e48e3a5170`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:22:40.214603 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:40.272086 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1477/2919] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:22:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:22:41.302710 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:41.304278 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:41.380480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def593-f7de-4b4a-a98a-5d343199d83a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:41.388664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:41.389297 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:41.389584 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:41.389898 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:41.393145 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:41.405484 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:41.407359 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:41.417840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1443/2920] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:22:50.263558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=25 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:51.413445 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=32 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:22:56.583434 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:22:56.585791 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:56.672905 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3e1a73-f48c-4c1c-9b90-6806a4d64e64 None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:22:56.682684 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:22:56.683426 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:22:56.683721 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:22:56.683993 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:22:56.685953 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:22:56.700178 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:22:56.702046 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:22:56.716630 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1478/2921] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:22:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:06.708858 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=34 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:20.056726 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:20.059182 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:20.117934 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80cfa-139b-4e85-9654-59c3deedfc56 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:20.121913 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:20.123129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:20.123417 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:20.123697 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:20.125491 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:20.187505 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:20.188951 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:20.198247 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1444/2922] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:26.664804 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:26.675103 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566007d-5361-4d4d-9c91-cf9c9a3fe99d service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:26.678739 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:26.679355 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:26.679633 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:26.679903 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:26.681609 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:26.746405 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:26.748923 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:26.759060 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1479/2923] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:30.193932 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=48 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:34.217698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:34.218701 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:34.234026 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc2be-d214-4b13-a462-d93efa747ebc service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:34.239129 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:34.239803 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:34.240097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:34.240375 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:34.242293 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:34.254827 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:34.256431 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:34.267793 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1445/2924] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:23:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:23:34.429187 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:34.435173 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:34.458646 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a057d9d-8e52-4a9f-9f41-e2cbde86084f service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:34.463350 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:34.463973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:34.464253 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:34.464531 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] PATH_INFO: `/v3/users/9352fbf7d98b40d7954ce1ff3d968761` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:34.470236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9352fbf7d98b40d7954ce1ff3d968761)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:34.472210 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:34.678702 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9352fbf7d98b40d7954ce1ff3d968761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:34.686536 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9352fbf7d98b40d7954ce1ff3d968761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:34.687221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9352fbf7d98b40d7954ce1ff3d968761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:34.687842 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9352fbf7d98b40d7954ce1ff3d968761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:34.701212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9352fbf7d98b40d7954ce1ff3d968761'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:34.813460 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1480/2925] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:23:34 2022] DELETE /identity/v3/users/9352fbf7d98b40d7954ce1ff3d968761 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:23:35.077135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:35.091263 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc0d75-893b-4785-87fa-f7cf1da5b493 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:35.150143 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:35.150776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:35.151054 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:35.151326 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] PATH_INFO: `/v3/projects/ba3bd03ea50c4553b9b98b61a4d835dd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:35.152422 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3bd03ea50c4553b9b98b61a4d835dd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:35.155112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:35.269876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3bd03ea50c4553b9b98b61a4d835dd', 'request_id': 'req-065d062d-749b-42ad-baca-85f7fa071091'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.276132 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3bd03ea50c4553b9b98b61a4d835dd', 'request_id': 'req-065d062d-749b-42ad-baca-85f7fa071091'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.282548 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3bd03ea50c4553b9b98b61a4d835dd', 'request_id': 'req-065d062d-749b-42ad-baca-85f7fa071091'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.364099 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] The token cache is being invalidate because project ba3bd03ea50c4553b9b98b61a4d835dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:23:35.364391 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.365023 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.366565 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1446/2926] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:23:35 2022] DELETE /identity/v3/projects/ba3bd03ea50c4553b9b98b61a4d835dd => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:23:35.370747 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:35.372189 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:35.385563 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6477ead-b597-4365-9d16-03f7d7dff9dc admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:35.442436 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:35.443195 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:35.443346 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:35.443620 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] PATH_INFO: `/v3/users/c3b49ba2b14545acbb9cc7cd9511a2aa` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:35.444773 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3b49ba2b14545acbb9cc7cd9511a2aa)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:35.446760 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:35.453825 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:35.654515 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3b49ba2b14545acbb9cc7cd9511a2aa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.661553 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3b49ba2b14545acbb9cc7cd9511a2aa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.662287 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3b49ba2b14545acbb9cc7cd9511a2aa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.667623 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3b49ba2b14545acbb9cc7cd9511a2aa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.675291 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3b49ba2b14545acbb9cc7cd9511a2aa'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:35.755167 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1481/2927] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:23:35 2022] DELETE /identity/v3/users/c3b49ba2b14545acbb9cc7cd9511a2aa => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:23:36.018810 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:23:36.020372 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:23:36.034112 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d062d-749b-42ad-baca-85f7fa071091 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:23:36.095201 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:23:36.095829 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:23:36.096163 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:23:36.096442 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] PATH_INFO: `/v3/projects/0b43e37e5945406ca905cecad78e76c4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:23:36.097567 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b43e37e5945406ca905cecad78e76c4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:23:36.099506 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:23:36.120983 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:36.202272 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b43e37e5945406ca905cecad78e76c4', 'request_id': 'req-a361c622-a31a-45e9-b10d-a1a53d1debf3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:36.208482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b43e37e5945406ca905cecad78e76c4', 'request_id': 'req-a361c622-a31a-45e9-b10d-a1a53d1debf3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:36.214043 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b43e37e5945406ca905cecad78e76c4', 'request_id': 'req-a361c622-a31a-45e9-b10d-a1a53d1debf3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:36.329884 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] The token cache is being invalidate because project 0b43e37e5945406ca905cecad78e76c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:23:36.330573 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:36.330875 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:23:36.332425 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1447/2928] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:23:36 2022] DELETE /identity/v3/projects/0b43e37e5945406ca905cecad78e76c4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:23:45.751547 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:23:46.326660 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:01.451572 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:01.452990 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:01.524064 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103b3902-6c94-4d42-879c-e711ef7b778a admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:01.584896 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:01.585520 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:01.585843 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:01.586130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:01.587906 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:01.599212 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:01.600797 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a6ef26-fe92-4d57-9e9f-f3fc69f77272 service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:01.610823 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1482/2929] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:24:11.606532 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=58 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:24:44.295977 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:44.297393 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:44.355692 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361c622-a31a-45e9-b10d-a1a53d1debf3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:44.359318 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb22d91ffd224b6aa9093a440d5e4925', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:44.359926 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:44.360205 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:44.360482 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:44.362352 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:44.423084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:44.424532 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:44.433693 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1448/2930] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:24:50.338585 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] REQUEST_METHOD: `POST` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:50.339383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:50.339383 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:50.363016 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.auth.core [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] MFA Rules not processed for user `b5b9eb7e61704da582a326e48e3a5170`. Rule list: `[]` (Enabled: `True`). {{(pid=81621) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 01:24:50.364473 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:50.372790 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1483/2931] 10.222.0.52 () {58 vars in 958 bytes} [Tue Aug 30 01:24:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 01:24:50.748183 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:24:50.749699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:50.808775 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e321a1-e6f0-4bbb-a887-c19085b5b312 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:24:50.812712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:24:50.813035 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:24:50.813313 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:24:50.813588 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:24:50.815332 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:24:50.826433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:24:50.827840 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:24:50.836407 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1449/2932] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:24:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:25:00.356047 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=3 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:00.832368 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=74 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:25:12.608894 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:25:12.610390 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:12.687606 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73e8a6-7a44-47af-aeb2-41318678867e None None] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:25:12.693298 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e1a623f1fc434449a76106e74756e35c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:25:12.693938 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:25:12.694221 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:25:12.694493 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:25:12.696236 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:25:12.709005 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:25:12.710676 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:25:12.722140 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1484/2933] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:25:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:25:22.717234 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=34 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:42.875403 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:42.876876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:42.888097 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278874e-650b-4f03-8c91-5f1f5d53abd4 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:42.891963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:42.892592 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:42.892876 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:42.893187 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:42.894936 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:42.907022 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:42.908499 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:42.919206 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1450/2934] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:26:43.193943 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:43.204502 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aa6162-f022-405f-b5e0-281709e3cf33 service nova] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:43.209988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:43.209988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:43.209988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:43.209988 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:43.211509 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:43.226051 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:43.227485 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:43.239163 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1485/2935] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:26:44.421698 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:44.431084 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33918f-3561-4ecc-9b04-573bc5950a1d service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:44.434762 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:44.435354 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:44.435634 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:44.435910 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:44.437665 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:44.451135 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:44.452555 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:44.464663 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1451/2936] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:26:53.233873 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=6 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:54.458893 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:26:58.952784 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:26:58.961596 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc528c3-43d3-438d-8189-fe688d98a6a1 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:26:58.966306 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '851cf4bd4efd448f8fc04563775de1ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:26:58.966911 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:26:58.967207 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:26:58.967486 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:26:58.969199 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:26:59.032992 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:26:59.034494 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:26:59.044069 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1486/2937] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:06.402074 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:06.403699 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:06.418606 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508b9f2f-e3cf-4658-9ee1-ea73ed5548cc service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:06.423877 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:06.424488 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:06.424776 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:06.425061 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:06.426854 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:06.439411 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:06.441012 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:06.453790 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1452/2938] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:27:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:27:06.567363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:06.568893 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:06.587501 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29aeaa-3056-45bf-9527-83dc6c63fad5 service neutron] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:06.598558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:06.598558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:06.598558 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:06.599068 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] PATH_INFO: `/v3/users/b5b9eb7e61704da582a326e48e3a5170` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:06.599973 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5b9eb7e61704da582a326e48e3a5170)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:06.602261 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:06.820748 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5b9eb7e61704da582a326e48e3a5170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:06.827910 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5b9eb7e61704da582a326e48e3a5170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:06.828578 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5b9eb7e61704da582a326e48e3a5170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:06.829179 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5b9eb7e61704da582a326e48e3a5170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:06.842006 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5b9eb7e61704da582a326e48e3a5170'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:06.927537 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1487/2939] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:06 2022] DELETE /identity/v3/users/b5b9eb7e61704da582a326e48e3a5170 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:07.202868 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:07.214546 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39380e8e-1757-472e-bc9a-25a003efa5c0 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:07.275392 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:07.276045 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:07.276331 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:07.276611 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] PATH_INFO: `/v3/projects/3068817b4ca34beca8bd4da5409e58ef` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:07.277784 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3068817b4ca34beca8bd4da5409e58ef)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:07.280821 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:07.407867 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3068817b4ca34beca8bd4da5409e58ef', 'request_id': 'req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.414522 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3068817b4ca34beca8bd4da5409e58ef', 'request_id': 'req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.421114 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3068817b4ca34beca8bd4da5409e58ef', 'request_id': 'req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.481232 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] The token cache is being invalidate because project 3068817b4ca34beca8bd4da5409e58ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:07.481450 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.482124 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.486664 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1453/2940] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:07 2022] DELETE /identity/v3/projects/3068817b4ca34beca8bd4da5409e58ef => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:07.487885 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:07.489367 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:07.502487 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18286-6754-49f4-9ab4-f3e2b805a253 admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:07.558376 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:07.559072 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:07.559385 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:07.559667 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] PATH_INFO: `/v3/users/73d80c58ebfe4e589d3887353b26faf3` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:07.560820 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] RBAC: Authorizing `identity:delete_user(user_id=73d80c58ebfe4e589d3887353b26faf3)` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:07.562801 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:07.568305 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:07.752130 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73d80c58ebfe4e589d3887353b26faf3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.761336 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73d80c58ebfe4e589d3887353b26faf3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.762371 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73d80c58ebfe4e589d3887353b26faf3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.763363 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73d80c58ebfe4e589d3887353b26faf3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.778927 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.notifications [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73d80c58ebfe4e589d3887353b26faf3'} {{(pid=81621) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:07.870372 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1488/2941] 10.222.0.52 () {58 vars in 1224 bytes} [Tue Aug 30 01:27:07 2022] DELETE /identity/v3/users/73d80c58ebfe4e589d3887353b26faf3 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 01:27:08.135081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:27:08.135081 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:27:08.150832 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86598fe6-609a-47b2-a5bc-00ef6e5b3bdd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:27:08.213480 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efbdbf875e6a427e9e8f14457c4de416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'dae85e39d46d422d97a780b67b09cea1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:27:08.214149 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:27:08.214433 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:27:08.214712 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] PATH_INFO: `/v3/projects/561828cfc1c64d8b89e7e13f39cd6911` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:27:08.215859 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561828cfc1c64d8b89e7e13f39cd6911)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:27:08.217879 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:27:08.240973 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:08.361402 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561828cfc1c64d8b89e7e13f39cd6911', 'request_id': 'req-28e1983e-8f5c-447a-afb2-f813875d0b30'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.369373 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561828cfc1c64d8b89e7e13f39cd6911', 'request_id': 'req-28e1983e-8f5c-447a-afb2-f813875d0b30'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.380328 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561828cfc1c64d8b89e7e13f39cd6911', 'request_id': 'req-28e1983e-8f5c-447a-afb2-f813875d0b30'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.503031 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] The token cache is being invalidate because project 561828cfc1c64d8b89e7e13f39cd6911 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 01:27:08.503667 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.503963 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-28e1983e-8f5c-447a-afb2-f813875d0b30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 01:27:08.505466 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1454/2942] 10.222.0.52 () {58 vars in 1233 bytes} [Tue Aug 30 01:27:08 2022] DELETE /identity/v3/projects/561828cfc1c64d8b89e7e13f39cd6911 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 01:27:17.865914 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:27:18.499656 dv-cinder-815712-11-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 01:29:08.446345 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Authenticating user token {{(pid=81621) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 01:29:08.447741 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:08.597277 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71823dad-95fb-4c72-bc19-f3af1df5bebf admin admin] Validating token access rules against request {{(pid=81621) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 01:29:08.658129 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d22a2e2f374aa6bba35838ebde3b4c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8641bb2fa3db4ff79d2bb5fc288a6c81', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81621) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 01:29:08.658742 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] REQUEST_METHOD: `GET` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 01:29:08.659025 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] SCRIPT_NAME: `/identity` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 01:29:08.659303 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81621) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 01:29:08.661057 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.fernet_utils [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81621) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 01:29:08.672929 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 01:29:08.674566 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5ac76a-0f9b-47e1-a795-f4e2c50796ef service placement] RBAC: Authorization granted {{(pid=81621) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 01:29:08.684734 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: [pid: 81621|app: 0|req: 1489/2943] 10.222.0.52 () {60 vars in 1365 bytes} [Tue Aug 30 01:29:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 01:29:18.680356 dv-cinder-815712-11-iscsi devstack@keystone.service[81621]: DEBUG dbcounter [-] [81621] Writing DB stats keystone:SELECT=89 {{(pid=81621) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}